Road surface damage analysis device and road surface damage analysis method

Abstract

The purpose of the present invention is to provide a technology capable of preventing mistaken analysis in which glare is assumed to be damage. This road surface damage analysis device comprises: an image acquisition unit that acquires an image of a road surface captured from a vehicle through window glass; a determination unit that determines whether glare on the window glass resulting from sunlight is included in the image; and an analysis unit. The analysis unit analyzes the damage state of the road surface, using a rejection unit that rejects at least a portion of the image, which has been determined to include the glare, and on the basis of the image other than the at least portion thereof.

Description

Road surface damage analysis device and road surface damage analysis method

[0001] The present disclosure relates to a road surface damage analysis device and a road surface damage analysis method.

[0002] Various technologies have been proposed for devices that analyze the state of road deterioration from images obtained by photographing the road surface. For example, Patent Document 1 proposes a technology that detects lane marks with high accuracy and narrows the detection area to detect cracks. For example, Patent Document 2 proposes a technology that acquires the date, location, time, and direction at which an image was taken, calculates the direction in which a shadow of a structure may be cast on the road surface in the image, and suppresses erroneous detection of deterioration based on the direction of deterioration on the road surface and the direction in which the shadow may be cast.

[0003] JP 2019-144608 A International Publication No. 2021 / 200038

[0004] However, with conventional technology, there are cases where the image contains reflections of sunlight on the windshield window, and in such cases, the reflections may be erroneously analyzed as damage.

[0005] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technology that can prevent reflections from being mistakenly analyzed as damage.

[0006] The road surface damage analysis device according to the present disclosure includes an image acquisition unit that acquires an image of the road surface photographed from a vehicle through a window glass, a determination unit that determines whether the image contains reflections of sunlight on the window glass, a rejection unit that rejects at least a portion of the image that is determined to contain reflections, and an analysis unit that analyzes the damage state of the road surface based on the image other than at least the portion.

[0007] According to the present disclosure, at least a portion of an image determined to include a window glass reflection is rejected, and the damage state of the road surface is analyzed based on the image other than the at least that portion. With this configuration, it is possible to prevent the reflection from being erroneously analyzed as damage.

[0008] The objects, features, aspects and advantages of the present disclosure will become more apparent from the following detailed description and the accompanying drawings.

[0009] 1 is a block diagram showing the configuration of a road damage analysis device according to embodiment 1. FIG. 2 is a flowchart showing the operation of the road damage analysis device according to embodiment 1. FIG. 3 is a block diagram showing the configuration of a road damage analysis device according to embodiment 2. FIG. 4 is a flowchart showing the operation of the road damage analysis device according to embodiment 2. FIG. 5 is a block diagram showing the configuration of a road damage analysis device according to embodiment 3. FIG. 6 is a flowchart showing the operation of the road damage analysis device according to embodiment 3. FIG. 7 is a diagram showing an example of the arrangement of a plurality of imaging devices. FIG. 8 is a flowchart showing the operation of the road damage analysis device according to embodiment 4. FIG. 9 is a flowchart showing the operation of the road damage analysis device according to embodiment 5. FIG. 10 is a flowchart showing the operation of the road damage analysis device according to embodiment 6. FIG. 11 is a block diagram showing the hardware configuration of a driving assistance device according to another modified example. FIG. 12 is a block diagram showing the hardware configuration of a driving assistance device according to another modified example.

[0010] <Embodiment 1> Fig. 1 is a block diagram showing the configuration of a road surface damage analysis device 1 according to Embodiment 1. The road surface damage analysis device 1 in Fig. 1 includes an image acquisition unit 11, an image storage unit 12 which is a storage unit, a brightness acquisition unit 13, a reflection determination unit 14 which is a determination unit, a reflection image rejection unit 15 which is a rejection unit, a vehicle information acquisition unit 16, and an analysis unit 17. In the following, the components of the road surface damage analysis device 1 will be described as being all provided in a vehicle, however, as will be described later, this is not limited to this.

[0011] The vehicle is provided with a camera (not shown) that captures an image of the road surface through a window such as a windshield from the vehicle. The camera may be, for example, an infrared camera or an RGB camera.

[0012] The image acquisition unit 11 acquires an image of the road surface captured by the imaging device from the imaging device. The image acquisition unit 11 may be an interface for the imaging device or may include the imaging device.

[0013] The image storage unit 12 stores the image acquired by the image acquisition unit 11 as a past image. The image storage unit 12 may be, for example, a dynamic random access memory (DRAM), a digital versatile disk (HDD), or other storage device. Note that the image storage unit 12 is not essential in the first embodiment.

[0014] The brightness acquisition unit 13 acquires the brightness of the surroundings of the vehicle illuminated by sunlight. The brightness acquisition unit 13 may be an interface for an illuminance sensor that detects the brightness of the surroundings of the vehicle, or may include an illuminance sensor. Furthermore, when the weather information for the area where the vehicle is traveling includes a cloudiness level, the brightness acquisition unit 13 may acquire the brightness based on the cloudiness level.

[0015] The reflection determination unit 14 determines whether reflection of sunlight on window glass is included in the image acquired by the image acquisition unit 11. Reflection is an image that appears in the image of the image capture device due to sunlight reflected at least by window glass. Reflection includes, for example, an image that appears in the image of the image capture device due to sunlight being reflected by an object and then by window glass.

[0016] In the first embodiment, reflection determination unit 14 determines that the image does not contain a reflection when the brightness acquired by brightness acquisition unit 13 is equal to or less than a threshold value, and determines that the image contains a reflection when the brightness is greater than the threshold value. Note that in Fig. 1, brightness acquisition unit 13 is provided separately from reflection determination unit 14, but reflection determination unit 14 may include brightness acquisition unit 13 that acquires brightness, which is information for determination.

[0017] The reflection image rejection unit 15 rejects at least a portion of an image determined to contain a reflection. In the first embodiment, the reflection image rejection unit 15 rejects all portions of an image determined to contain a reflection, but may also reject only a portion of an image determined to contain a reflection as in the fifth embodiment. The reflection image rejection unit 15 outputs the images that were not rejected to the analysis unit 17.

[0018] The vehicle information acquisition unit 16 acquires vehicle information including the vehicle position and the azimuth angle of the vehicle's forward direction relative to north. If the azimuth angle is based on a point other than north, the azimuth angle may be appropriately converted so that the azimuth angle is based on north. The vehicle information acquisition unit 16 may be an interface for a satellite positioning sensor such as a GPS (Global Positioning System) that detects the latitude and longitude indicating the vehicle position, or may include a satellite positioning sensor. In the first embodiment, the vehicle information acquisition unit 16 is not essential.

[0019] The analysis unit 17 analyzes the damage state of the road surface based on images other than at least a portion of the images determined to contain reflections among the images acquired by the image acquisition unit 11. In the first embodiment, the analysis unit 17 analyzes the damage state of the road surface based on images other than the images determined to contain reflections, but the analysis unit 17 may also analyze the damage state of the road surface based on images other than a portion, as in the fifth embodiment. Hereinafter, the image to be analyzed by the analysis unit 17 may also be referred to as the "image to be analyzed."

[0020] The analysis unit 17 may analyze the road surface damage state by determining whether or not there is road surface damage in the analysis target image, and may further determine the type of the damage. The analysis unit 17 may analyze the road surface damage state by performing machine learning (training) on ​​the analysis target image, or may analyze the road surface damage state by performing image analysis of the analysis target image other than machine learning.

[0021] The analysis unit 17 may also generate analysis result information by linking the analysis result of the road surface damage state with the vehicle position and azimuth angle acquired by the vehicle information acquisition unit 16. The analysis result information may be used to search for a route to the vehicle's destination, or may be transmitted to a server or the like.

[0022] 2 is a flowchart showing the operation (i.e., the road surface damage analysis method) of the road surface damage analysis device 1 according to the present embodiment 1. Note that this operation is performed while the vehicle is traveling.

[0023] First, in step S1, the vehicle information acquisition unit 16 acquires vehicle information. In step S2, the image acquisition unit 11 acquires an image of the road surface from the photographing device and stores it in the image storage unit 12. In step S3, the brightness acquisition unit 13 acquires the brightness around the vehicle.

[0024] In step S4, reflection determination unit 14 determines whether the image contains reflections on the window glass caused by sunlight. In the first embodiment, reflection determination unit 14 determines whether the image contains reflections based on whether the brightness acquired by brightness acquisition unit 13 is equal to or less than a threshold. If it is determined that the image contains reflections, the process proceeds to step S5, and if it is determined that the image does not contain reflections, the process proceeds to step S6.

[0025] In step S5, the reflection image rejection unit 15 rejects the image determined to include a reflection, and then the operation of FIG.

[0026] In step S6, the reflected image rejection unit 15 outputs the images that were not rejected to the analysis unit 17, and the analysis unit 17 analyzes the damage state of the road surface based on the images. After that, the operation in FIG. 2 ends.

[0027] Summary of First Embodiment As described above, the road surface damage analysis device 1 according to the first embodiment discards images determined to include reflections on window glass, and analyzes the state of road surface damage based on images other than the discarded images. This configuration makes it possible to prevent reflections from being erroneously analyzed as damage.

[0028] Furthermore, in the first embodiment, the reflection determination unit 14 determines whether or not the image contains reflections based on whether or not the brightness acquired by the brightness acquisition unit 13 is equal to or less than a threshold value. With this configuration, the load of the calculation process can be made relatively low.

[0029] 3 is a block diagram showing the configuration of a road surface damage analysis device 1 according to this embodiment 2. In the following, among the components according to this embodiment 2, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components will be mainly described.

[0030] The configuration of Figure 3 is the same as the configuration of Figure 1, except that a time acquisition unit 21 is provided instead of the brightness acquisition unit 13, and vehicle information acquired by the vehicle information acquisition unit 16 is output to the reflection determination unit 14.

[0031] The time acquisition unit 21 acquires the standard time at the location of the vehicle as the time. The time acquisition unit 21 may be an interface for a timer or may include a timer.

[0032] The reflection determination unit 14 determines the solar azimuth angle based on the vehicle position included in the vehicle information acquired by the vehicle information acquisition unit 16 and the time acquired by the time acquisition unit 21. In the second embodiment, the reflection determination unit 14 first applies the latitude φ and longitude λ (positive values ​​for east longitude and negative values ​​for west longitude) included in the vehicle position and the time t, which is standard time, to the following equations (1) and (2) to determine the hour angle H and solar altitude h.

[0033] H=θ0+t×1.0027379+λ−α...(1)

[0034] h=arcsin(sinφsinδ+cosφcosδcosH)...(2)

[0035] Note that θ0 is Greenwich apparent sidereal time in universal time, α is the solar right ascension, and δ is the solar apparent declination. θ0, α, and δ may be values ​​from a predetermined chronology, such as those published by the National Astronomical Observatory, or other values ​​(e.g., values ​​derived from various mathematical formulas). When calculating the solar azimuth angle to the degree, values ​​calculated by proportionally dividing the value of the day and the value of the next day may be used for α and δ. The effect of atmospheric refraction may also be ignored.

[0036] If the hour angle H is a negative value, the reflection determination unit 14 adds 24 [h] to the hour angle H, and if the hour angle H exceeds 24 [h], it subtracts 24 [h] from the hour angle H. The reflection determination unit 14 applies the values ​​obtained above to the following equations (3) and (4) to find the solar azimuth angle A.

[0037] sinA=-cosδsinH / cosh...(3)

[0038] cosA=(cosφsinδ−sinφcosδcosH) / cosh...(4)

[0039] The solar azimuth angle A is the angle measured clockwise from the vehicle center from north (0 degrees) to the direction of the sun. If the value of formula (4) is negative, the reflection determination unit 14 adds 180 degrees to A, and if the value of formula (3) is negative and the value of formula (4) is positive, the reflection determination unit 14 adds 360 degrees to A.

[0040] The reflection determination unit 14 determines whether or not a reflection is included in the image based on the calculated solar azimuth angle and the vehicle azimuth angle included in the vehicle information acquired by the vehicle information acquisition unit 16. In the second embodiment, as an example, the reflection determination unit 14 determines that a reflection is included in the image when the difference (absolute value) between the solar azimuth angle and the vehicle azimuth angle is equal to or smaller than a threshold value, and determines that a reflection is not included in the image when the difference is greater than the threshold value.

[0041] The threshold value may be changed based on the angle of view of the imaging device. In Fig. 3, the time acquisition unit 21 and the vehicle information acquisition unit 16 are provided separately from the reflection determination unit 14, but the reflection determination unit 14 may include the time acquisition unit 21 that acquires the time, which is information for determination, and the vehicle information acquisition unit 16 that acquires the vehicle information, which is information for determination. Furthermore, the reflection determination unit 14 may determine that a reflection is included in the image when the difference (absolute value) between the solar azimuth angle and the vehicle azimuth angle is within a certain range.

[0042] <Operation> Figure 4 is a flowchart showing the operation of the road surface damage analysis device 1 according to the second embodiment. The operation in Figure 4 is the same as the operation in Figure 2, except that steps S1, S3, and S4 are replaced by steps S1a, S3a, and S4a. Therefore, the following description will mainly focus on steps S1a, S3a, and S4a.

[0043] In step S1a, the vehicle information acquisition unit 16 acquires vehicle information, and the time acquisition unit 21 acquires the time.

[0044] In step S3a, the reflection determination unit 14 determines the solar azimuth angle based on the vehicle information and the time.

[0045] In step S4a, the reflection determination unit 14 determines whether or not the image contains a reflection on the window glass caused by sunlight. In the second embodiment, the reflection determination unit 14 determines whether or not the image contains a reflection based on the solar azimuth angle and the vehicle azimuth angle included in the vehicle information. If it is determined that the image contains a reflection, the process proceeds to step S5, and if it is determined that the image does not contain a reflection, the process proceeds to step S6.

[0046] Summary of Second Embodiment According to the road surface damage analysis device 1 of the second embodiment, the reflection determination unit 14 calculates the solar azimuth angle based on the vehicle position and time, and determines whether or not a reflection is included in the image based on the solar azimuth angle and the azimuth angle of the vehicle. With this configuration, it is possible to determine whether or not a reflection is included in the image without providing the brightness acquisition unit 13 of FIG.

[0047] <Embodiment 3> Fig. 5 is a block diagram showing the configuration of a road surface damage analysis device 1 according to Embodiment 3. Hereinafter, among the components according to Embodiment 3, components that are the same as or similar to the components described above will be given the same or similar reference numerals, and different components will be mainly described.

[0048] The configuration of Figure 5 is the same as the configuration of Figure 1, except that the brightness acquisition unit 13 is omitted and the image acquired by the image acquisition unit 11 is output to the reflection determination unit 14 via the image storage unit 12.

[0049] The reflection determination unit 14 determines whether or not a reflection is included in an image based on the learning results of past images and the current image. The learning results of past images in the first and second examples will be described below.

[0050] The learning result of the past image in the first example is a trained reflection detection model obtained by performing machine learning (training) on ​​the past image as learning data to determine whether or not the image contains a reflection. In this case, the reflection determination unit 14 applies the current image to the reflection detection model and obtains, as an output from the reflection detection model, whether or not the current image contains a reflection.

[0051] The learning result of the past image in the second example is a learned direction detection model obtained by performing machine learning (training) on ​​the sun direction relative to the forward direction of the vehicle in a planar view using the past image as learning data. In this case, the reflection determination unit 14 applies the current image to the reflection detection model, obtains the sun direction as an output from the direction detection model, and determines whether or not a reflection is included in the image based on the sun direction.

[0052] Machine learning includes at least one of supervised learning, unsupervised learning, and reinforcement learning. In this specification, for example, "at least one of A, B, C, ..., and Z" means any one of all combinations of one or more types extracted from the group A, B, C, ..., and Z. The reflection determination unit 14 may externally acquire learning results of past images, or may generate learning results by performing learning itself.

[0053] <Operation> Figure 6 is a flowchart showing the operation of the road surface damage analysis device 1 according to the third embodiment. The operation in Figure 6 is the same as the operation in Figure 2, except that step S3 is deleted and step S4 is changed to step S4b. Therefore, the following description will mainly focus on step S4b.

[0054] In step S4b, reflection determination unit 14 determines whether the image contains reflections on the window glass caused by sunlight. In the third embodiment, reflection determination unit 14 determines whether the image contains reflections based on the learning results of past images and the current image. If it is determined that the image contains reflections, the process proceeds to step S5, and if it is determined that the image does not contain reflections, the process proceeds to step S6.

[0055] Summary of Third Embodiment According to the road surface damage analysis device 1 of the third embodiment, the reflection determination unit 14 determines whether or not a reflection is included in an image based on the learning results of past images and the current image. With this configuration, it is possible to determine whether or not a reflection is included in an image essentially from the image alone, without providing the brightness acquisition unit 13 of Fig. 1 and the time acquisition unit 21 of Fig. 3.

[0056] <Fourth Embodiment> The block diagram of the road surface damage analysis device 1 according to this fourth embodiment is the same as the block diagram of the road surface damage analysis device 1 according to the second embodiment in Figure 3. However, the block diagram of the road surface damage analysis device 1 according to this fourth embodiment may be the same as the block diagram of the road surface damage analysis device 1 according to the first and third embodiments in Figures 1 and 5. Hereinafter, among the components according to this fourth embodiment, components that are the same as or similar to the components described above will be assigned the same or similar reference numerals, and different components will be mainly described.

[0057] In the fourth embodiment, a plurality of imaging devices are provided on a vehicle. FIG. 7 is a diagram showing an example of the arrangement of a plurality of imaging devices, including a first imaging device 31a and a second imaging device 31b. In the example of FIG. 7, the first imaging device 31a is provided in the front portion of the vehicle, and the imaging direction of the first imaging device 31a is set to face forward of the vehicle. On the other hand, the second imaging device 31b is provided in the rear portion of the vehicle, and the imaging direction of the second imaging device 31b is set to face backward of the vehicle. In other words, the first imaging device 31a and the second imaging device 31b are provided point-symmetrically with respect to the center of the vehicle. Note that the number of the plurality of imaging devices may be three or more, as long as at least two of the three or more imaging devices are provided point-symmetrically with respect to the center of the vehicle.

[0058] The image acquisition unit 11 acquires multiple images from multiple image capture devices, each capturing a plurality of images, and stores the multiple images in the image storage unit 12. The reflection determination unit 14 performs a reflection determination for each of the multiple images while appropriately offsetting the azimuth angle of the vehicle based on the clockwise angle from the front of the vehicle (0 degrees) to the direction in which the image capture device is located, centered on the vehicle. For example, the reflection determination unit 14 performs a reflection determination for the image captured by the first image capture device 31a without offsetting the azimuth angle of the vehicle. On the other hand, the reflection determination unit 14 offsets the azimuth angle of the vehicle by 180 degrees and then performs a reflection determination for the image captured by the second image capture device 31b.

[0059] <Operation> Figure 8 is a flowchart showing the operation of the road surface damage analysis device 1 according to the fourth embodiment. The operation in Figure 8 is the same as the operation in Figure 4, except that steps S2 and S4a are replaced with steps S2c and S4c. Therefore, steps S2c and S4c will be mainly described below.

[0060] In step S2 c , the image acquisition unit 11 acquires a plurality of images of the road surface from a plurality of photographing devices and stores them in the image storage unit 12 .

[0061] In step S4c, the reflection determination unit 14 determines whether or not a reflection is included in each of the multiple images, similar to step S4a. For images determined to include a reflection, the process of step S5 is performed, and for images determined not to include a reflection, the process of step S6 is performed.

[0062] Summary of Fourth Embodiment According to the road surface damage analysis device 1 of the fourth embodiment, the image acquisition unit 11 acquires a plurality of images from a plurality of image capture devices, each capturing a plurality of images. With this configuration, the time during which the road surface damage state cannot be analyzed because the reflected image rejection unit 15 rejects images can be reduced by acquiring a plurality of images.

[0063] In the fourth embodiment, the plurality of image capturing devices are provided point-symmetrically with respect to the center of the vehicle. With this configuration, it is possible to reduce the time during which the road surface damage state cannot be analyzed using as few image capturing devices as possible.

[0064] <Fifth Embodiment> The block diagram of the road surface damage analysis device 1 according to this fifth embodiment is the same as the block diagram of the road surface damage analysis device 1 according to the second embodiment in Fig. 3. Hereinafter, among the components according to this fifth embodiment, components that are the same as or similar to the components described above will be given the same or similar reference numerals, and different components will be mainly described.

[0065] In the same manner as in the second embodiment, in the fifth embodiment, the reflection determination unit 14 determines the solar azimuth angle based on the vehicle position included in the vehicle information acquired by the vehicle information acquisition unit 16 and the time acquired by the time acquisition unit 21. In the fifth embodiment, the reflection determination unit 14 determines the reflection portion of the image based on the difference θ' between the solar azimuth angle and the azimuth angle of the vehicle, and the angle of view θc of the image capturing device that captures the image.

[0066] The difference θ' is found by subtracting the vehicle's azimuth angle from the solar azimuth angle (i.e., difference θ' = solar azimuth angle - vehicle's azimuth angle). For example, if 0≦θ'<θc / 2 holds, the reflection determination unit 14 determines that the image contains a reflection, and determines that the right side of the image is the reflection portion. If -θc / 2<0≦θ'<0 holds, the reflection determination unit 14 determines that the image contains a reflection, and determines that the left side of the image is the reflection portion. If θ' is outside the above range, the reflection determination unit 14 determines that the image does not contain a reflection.

[0067] The reflection image rejection unit 15 rejects the reflection portion from an image determined to contain a reflection, and outputs the image from which the reflection portion has been rejected (removed) to the analysis unit 17. For example, for an image whose right side is determined to be a reflection portion, the reflection image rejection unit 15 outputs the portion of the image other than the right side, i.e., the portion from the center to the left side, to the analysis unit 17. For an image whose left side is determined to be a reflection portion, the reflection image rejection unit 15 outputs the portion of the image other than the left side, i.e., the portion from the center to the right side, to the analysis unit 17. For an image determined not to contain a reflection, the reflection image rejection unit 15 outputs the image as is to the analysis unit 17 without processing it.

[0068] The analysis unit 17 analyzes the damage state of the road surface based on the image other than the reflected portion. The image other than the reflected portion is an image without any reflected portion or an image from which the reflected portion has been removed. Note that, although two reflected portions are provided corresponding to two ranges of θ' in the above description, three or more reflected portions may be provided corresponding to three or more ranges of θ'.

[0069] <Operation> Figure 9 is a flowchart showing the operation of the road surface damage analysis device 1 according to the fifth embodiment. The operation in Figure 9 is the same as the operation in Figure 4, except that steps S4a, S5, and S6 are replaced by steps S4d, S5d, and S6d. Therefore, the following description will mainly focus on steps S4d, S5d, and S6d.

[0070] In step S4d, the reflection determination unit 14 determines whether or not a reflection is included in the image based on the solar azimuth angle and the vehicle azimuth angle included in the vehicle information. If a reflection is included in the image, the reflection determination unit 14 determines the reflection portion of the image. If it is determined that a reflection is included in the image, the process proceeds to step S5d, and if it is determined that a reflection is not included in the image, the process proceeds to step S6d.

[0071] In step S5d, the reflection image rejection unit 15 rejects the reflection portion from the image determined to contain a reflection.

[0072] In step S6d, the analysis unit 17 analyzes the damage state of the road surface based on the image of the portion other than the reflected portion, and then the operation of FIG.

[0073] Summary of Embodiment 5 According to the road surface damage analysis device 1 of Embodiment 5 described above, the reflected portion is determined based on the difference θ' between the solar azimuth angle and the vehicle azimuth angle and the angle of view θc of the imaging device, and the damage state of the road surface is analyzed based on the image other than the reflected portion. This configuration can prevent the reflected image rejection unit 15 from rejecting images, thereby reducing the time during which the road surface damage state cannot be analyzed due to such rejection.

[0074] <Embodiment 6> The block diagram of the road surface damage analysis device 1 according to this embodiment 6 is the same as the block diagram of the road surface damage analysis device 1 according to embodiment 3 in Fig. 5. Hereinafter, among the components according to this embodiment 5, components that are the same as or similar to the components described above will be assigned the same or similar reference numerals, and different components will be mainly described.

[0075] In the sixth embodiment, the reflection determination unit 14 generates a difference image, which is the difference between a past image stored in the image storage unit 12 and a current image acquired by the image acquisition unit 11, and determines whether or not the image contains a reflection based on the difference image. The difference image is generated by calculating the absolute value of the difference between the luminance value of the past image and the luminance value of the current image for each pixel element.

[0076] For example, the sun is far from the camera and moves little between image frames, so the absolute value is small. Also, the glare caused by sunlight moves little between image frames, so the absolute value is small. On the other hand, features around the vehicle (e.g., roads, roadside trees, roadside strips, etc.) are close to the camera and move a lot between image frames, so the absolute value is large.

[0077] For this reason, it is considered that the number of pixels whose absolute values ​​exceed the threshold in an image that includes a reflection will be smaller than the number of pixels whose absolute values ​​exceed the threshold in an image that does not include a reflection. Based on this, the reflection determination unit 14 according to the sixth embodiment determines that the current image includes a reflection when the number of pixels whose absolute values ​​exceed the threshold in the difference image is equal to or less than a certain number.

[0078] <Operation> Figure 10 is a flowchart showing the operation of the road surface damage analysis device 1 according to the sixth embodiment. The operation in Figure 10 is the same as the operation in Figure 6, except that step S3e is added and step S4b is changed to step S4e. For this reason, steps S3e and S4e will be mainly described below.

[0079] In step S3 e , the reflection determining unit 14 generates a difference image from the past image stored in the image storage unit 12 and the current image acquired by the image acquiring unit 11 .

[0080] In step S4e, the reflection determination unit 14 determines whether or not the image contains reflections on the window glass caused by sunlight. In the sixth embodiment, the reflection determination unit 14 determines whether or not the image contains reflections based on the difference image. If it is determined that the image contains reflections, the process proceeds to step S5, and if it is determined that the image does not contain reflections, the process proceeds to step S6.

[0081] Summary of Sixth Embodiment According to the road surface damage analysis device 1 of the sixth embodiment described above, the reflection determination unit 14 determines whether or not a reflection is included in an image based on the difference between a past image and a current image stored in the image storage unit 12. With this configuration, it is possible to determine whether or not a reflection is included in an image essentially from the image alone, without providing the brightness acquisition unit 13 of Fig. 1 and the time acquisition unit 21 of Fig. 3.

[0082] <Other Modifications> The image acquisition unit 11, the reflection determination unit 14, the reflection image rejection unit 15, and the analysis unit 17 shown in FIG. 1 are hereinafter referred to as the "image acquisition unit 11, etc." The image acquisition unit 11, etc. are realized by a processing circuit 81 shown in FIG. 11. That is, the processing circuit 81 includes the image acquisition unit 11 that acquires an image of the road surface captured from a vehicle through a window glass, the reflection determination unit 14 that determines whether the image contains reflections on the window glass caused by sunlight, the reflection image rejection unit 15 that rejects at least a portion of the image determined to contain reflections, and the analysis unit 17 that analyzes the damage state of the road surface based on the image other than the at least portion. The processing circuit 81 may be implemented by dedicated hardware, or may be implemented by a processor that executes a program stored in a memory. Examples of the processor include a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, and a DSP (Digital Signal Processor).

[0083] When the processing circuitry 81 is dedicated hardware, the processing circuitry 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each unit, such as the image acquisition unit 11, may be realized by a circuit in which the processing circuits are distributed, or the functions of each unit may be realized by a single processing circuit.

[0084] When the processing circuit 81 is a processor, the functions of the image acquisition unit 11 and the like are realized in combination with software and the like. Software and the like may include, for example, software, firmware, or software and firmware. The software and the like are written as a program and stored in memory. As shown in FIG. 12 , the processor 82 applied to the processing circuit 81 realizes the functions of each unit by reading and executing a program stored in memory 83. That is, the road surface damage analysis device 1 includes a memory 83 for storing a program that, when executed by the processing circuit 81, results in the following steps: acquiring an image of the road surface taken from a vehicle through a window glass; determining whether the image contains a reflection of sunlight on the window glass; discarding at least a portion of the image determined to contain the reflection; and analyzing the road surface damage state based on the image other than at least the portion. In other words, this program can be said to cause a computer to execute the procedures and methods of the image acquisition unit 11 and the like. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read Only Memory), a HDD (Hard Disk Drive), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD, a drive device for any of these, or any storage medium that will be used in the future.

[0085] The above describes a configuration in which each function of the image acquisition unit 11 and the like is realized either by hardware or software, etc. However, this is not limited to this, and a configuration in which part of the image acquisition unit 11 and the like is realized by dedicated hardware and another part is realized by software, etc. For example, the function of the image acquisition unit 11 can be realized by a processing circuit 81 as dedicated hardware, and the other functions can be realized by the processing circuit 81 as a processor 82 reading and executing programs stored in a memory 83.

[0086] As described above, the processing circuitry 81 can realize the above-mentioned functions by hardware, software, or a combination of these.

[0087] The road damage analysis device described above can also be applied to a road damage analysis device constructed as a system by appropriately combining vehicle devices such as a portable navigation device (PND), a navigation device, and a driver monitoring system (DMS), communication terminals including mobile devices such as a mobile phone, a smartphone, and a tablet, application functions installed on at least one of the vehicle devices and the communication terminal, and a server. In this case, the functions or components of the road damage analysis device described above may be distributed among the devices that construct the system, or may be centralized in one of the devices. For example, the image acquisition unit 11, the reflection determination unit 14, and the reflection image rejection unit 15 may be provided in the vehicle, and the analysis unit 17 may be provided in a server that can communicate with the vehicle.

[0088] It should be noted that the embodiments and modifications may be freely combined, and the embodiments and modifications may be modified or omitted as appropriate.

[0089] The above description is illustrative in all respects and is not restrictive. It is understood that countless variations not illustrated can be envisioned.

[0090] REFERENCE SIGNS LIST 1 Road surface damage analysis device, 11 Image acquisition unit, 12 Image storage unit, 14 Reflection determination unit, 15 Reflection image rejection unit, 17 Analysis unit, 31a First imaging device, 31b Second imaging device.

Claims

1. A road surface damage analysis device comprising: an image acquisition unit that acquires an image of a road surface taken from a vehicle through a window glass; a determination unit that determines whether the image contains reflections of sunlight on the window glass; a rejection unit that rejects at least a portion of the image that is determined to contain the reflections; and an analysis unit that analyzes the damage state of the road surface based on the image other than the at least one portion.

2. A road surface damage analysis device as described in claim 1, wherein the judgment unit determines whether the reflection is included in the image based on whether the brightness of the area around the vehicle illuminated by the sunlight is below a threshold.

3. A road surface damage analysis device as described in claim 1, wherein the determination unit calculates the solar azimuth angle based on the position of the vehicle and the time, and determines whether the reflection is included in the image based on the solar azimuth angle and the azimuth angle of the vehicle.

4. A road surface damage analysis device as described in claim 1, wherein the judgment unit determines whether the reflection is included in the image based on the learning results of the past image and the current image.

5. A road surface damage analysis device as described in any one of claims 1 to 4, wherein the image acquisition unit acquires the multiple images from multiple image capture devices provided on the vehicle, each capturing a multiple number of the images.

6. A road surface damage analysis device according to claim 5, wherein the plurality of image capture devices are provided point-symmetrically with respect to the center of the vehicle.

7. A road surface damage analysis device as described in claim 1, wherein the determination unit calculates a solar azimuth angle based on the position and time of the vehicle, and determines the reflected portion of the image based on the difference between the solar azimuth angle and the azimuth angle of the vehicle and the angle of view of the imaging device that captures the image, and the rejection unit rejects the reflected portion from the image that is determined to contain the reflection.

8. A road surface damage analysis device as described in claim 1, further comprising a memory unit that stores the past images, and the judgment unit judges whether the reflection is included in the image based on the difference between the past image stored in the memory unit and the current image.

9. A road surface damage analysis device according to any one of claims 1 to 8, wherein the image acquisition unit, the determination unit, and the rejection unit are provided in the vehicle, and the analysis unit is provided in a server.

10. A road surface damage analysis method comprising: acquiring an image of a road surface taken from a vehicle through a window; determining whether the image contains reflections of sunlight on the window glass; discarding at least a portion of the image determined to contain the reflections; and analyzing the damage state of the road surface based on the image other than the at least a portion.

Images