Road surface measurement system

Abstract

To efficiently generate a road surface unevenness model by using a reference point outside a measurement range when the reference point does not exist in the measurement range.SOLUTION: A road surface measurement system that generates a road surface unevenness model by synthesizing a plurality of pieces of image data obtained by capturing an image of a road surface by an imaging device while causing a measurement vehicle on which the imaging device is mounted to autonomously travel by feature point matching processing, the road surface measurement system including: Feature point matching processing is executed by using a plurality of pieces of image data including at least any one of a measurement range, one or a plurality of out-of-range reference points, and a connection region set between the measurement range and the one or the plurality of out-of-range reference points in the obtained image data.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present disclosure relates to a road surface measurement system. [Background technology]

[0002] Various techniques are used to generate composite data by synthesizing multiple image data pieces having overlapping regions. For example, Patent Document 1 discloses a control device that acquires multiple captured images including depth information of the traveling surface of a moving object captured by multiple stereo imaging means, matches feature points extracted from each stereo captured image, and overlaps the acquired multiple captured images to connect them in the width direction of the traveling surface. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-164138 Summary of the Invention [Problem to be solved by the invention]

[0004] Here, by enabling the device (measurement vehicle) used to collect image data to travel autonomously, it becomes possible to automatically collect the image data used to generate synthetic data. To enable the measurement vehicle to travel autonomously, it is necessary to move the measurement vehicle along a measurement route. For example, by utilizing a satellite system such as GPS (Global Positioning System), the measurement vehicle can travel autonomously along the measurement route. However, the measurement vehicle cannot travel autonomously in an environment where satellite signals cannot be received, such as inside a tunnel.

[0005] Furthermore, instead of utilizing a satellite system, if there is a predetermined reference point whose position is managed within the measurement range of the road surface, image data of the road surface within the measurement range can be synthesized using the reference point as a constraint condition to generate a road surface unevenness model. On the other hand, if the above-mentioned reference point does not exist within the measurement range of the road surface, it is necessary to set a reference point outside the measurement range (out-of-range reference point), take images so as to include the out-of-range reference point and the road surface within the measurement range, and synthesize the image data. In such a case, since extra areas outside the measurement range of the road surface are included, the amount of image data required to generate the road surface unevenness model increases, and there is a risk that the time required to generate the road surface unevenness model will be excessive.

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and an object of the present disclosure is to provide a road surface measurement system that can efficiently generate a road surface unevenness model by utilizing out-of-range reference points when no reference points exist within the measurement range. [Means for solving the problem]

[0007] In order to solve the above problem, according to one aspect of the present disclosure, A road surface measurement system is provided. [Effects of the Invention]

[0008] As described above, according to the present disclosure, when no reference points exist within the measurement range, a road surface unevenness model can be efficiently generated by utilizing reference points outside the range. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an overview of a road surface measurement system according to a first embodiment of the present disclosure. [Figure 2] 10A and 10B are explanatory diagrams showing out-of-range reference points, measurement ranges, and connection regions according to the embodiment. [Figure 3] 10A and 10B are explanatory diagrams showing out-of-range reference points, measurement ranges, and connection regions according to the embodiment. [Figure 4]10A and 10B are explanatory diagrams showing a measurement range, an out-of-range reference point, and a connection region according to the embodiment. [Figure 5] FIG. 2 is an explanatory diagram showing a measurement vehicle of the road surface measurement system according to the embodiment as viewed from the side. [Figure 6] FIG. 2 is an explanatory diagram illustrating a measurement vehicle of the road surface measurement system according to the embodiment, viewed from above. [Figure 7] FIG. 2 is a block diagram showing an example of the configuration of a road surface measurement system according to the embodiment. [Figure 8] 10 is a flowchart of a processing operation of a control device of a measurement vehicle of the road surface measurement system according to the embodiment. [Figure 9] 10 is a flowchart of an autonomous driving control process performed by a control device of a measurement vehicle of the road surface measurement system according to the embodiment. [Figure 10] 10 is a flowchart of a self-position estimation process performed by a control device of a measurement vehicle of the road surface measurement system according to the embodiment. [Figure 11] 2 is an explanatory diagram showing a capturing range of image data generated by the road surface measurement system according to the embodiment. FIG. [Figure 12] 10 is a flowchart of a model generation process performed by the information processing device of the road surface measurement system according to the embodiment. [Figure 13] 10 is a flowchart of a processing operation of a control device of a measurement vehicle of a road surface measurement system according to a second embodiment of the present disclosure. [Figure 14] 2 is an explanatory diagram showing a capturing range of image data generated by the road surface measurement system according to the embodiment. FIG. [Figure 15] 10 is a flowchart of a model generation process performed by the information processing device of the road surface measurement system according to the embodiment. [Figure 16] 10 is an explanatory diagram showing a synthesis process of image data by the information processing device of the road surface measurement system according to the embodiment. FIG. [Figure 17] 10 is a flowchart of a model generation process performed by an information processing device of a road surface measurement system according to a third embodiment of the present disclosure. [Figure 18]10 is an explanatory diagram showing a synthesis process of image data by the information processing device of the road surface measurement system according to the embodiment. FIG. [Figure 19] 10 is a flowchart of a processing operation of a control device of a measurement vehicle in a road surface measurement system according to a fourth embodiment of the present disclosure. [Figure 20] 10 is a flowchart of a model generation process performed by the information processing device of the road surface measurement system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same configurations are designated by the same reference numerals, and redundant description will be omitted.

[0011] <<1. First Embodiment>> <1-1. Overview of the road surface measurement system> First, an overview of a road surface measurement system according to an embodiment of the present disclosure will be described.

[0012] FIG. 1 is a diagram shown to explain an example of an application of the road surface measurement system of the present disclosure. 1, the road surface measurement system 1 includes a measurement vehicle 10 equipped with an imaging device, and an information processing device 100 communicatively connected to the measurement vehicle 10. The measurement vehicle 10 and the information processing device 100 are communicatively connected using, for example, cloud computing technology.

[0013] The measurement vehicle 10 captures an image of the road surface within a measurement range D of the road to be measured using an imaging device to generate image data Img_α, and transmits the generated image data Img_α to the information processing device 100 either sequentially or after the image capturing is completed. The information processing device 100 acquires the image data Img_α and generates a road surface unevenness model, which is three-dimensional data indicating the unevenness of the road surface, based on the acquired image data Img_α.

[0014] For example, the measurement vehicle 10 divides the measurement range D into three areas D1 to D3 and captures images of the road surface. The measurement vehicle 10 generates time-series image data Img_α while autonomously traveling through each of the areas D1 to D3. Adjacent areas D1 to D3 are set to partially overlap so that the entire measurement range D does not fall outside the imaging range.

[0015] The measurement vehicle 10 also has multiple imaging devices arranged in a direction (left-right direction) perpendicular to the direction of movement (front-back direction). The multiple imaging devices each capture images of the road surface while the measurement vehicle 10 autonomously travels through each of the areas D1 to D3, and generate multiple pieces of image data Img_α. The multiple imaging devices are installed so that the imaging ranges P1 to P6 of adjacent imaging devices partially overlap, so that the entirety of each of the areas D1 to D3 is not excluded from the imaging range.

[0016] Furthermore, the moving speed of the measurement vehicle 10 is set so that the time-series image data Img_α generated by each imaging device partially overlaps. That is, the imaging range at time t_n partially overlaps with the imaging range at the previous time t_n-1 and the imaging range at the next time t_n+1. However, for example, if the moving speed of the measurement vehicle is slow, the imaging range of a certain calculation cycle (time t_n) may overlap with the imaging range of the calculation cycle two cycles before that (times t_n-2, t_n-3,...), and the imaging range of the calculation cycle two cycles after that (times t_n+2, t_n+3,...).

[0017] 1 shows an example in which the road to be measured extends linearly and the measurement vehicle 10 travels straight, but the shape of the measurement object may be curved, and the example is not limited to the measurement vehicle 10 traveling straight. Furthermore, the division of the areas when measuring the measurement range D, the order in which the areas travel, and the route of the measurement vehicle 10 are not limited to the above example. Furthermore, the number of imaging devices is not particularly limited.

[0018] The information processing device 100 extracts feature points from the acquired multiple pieces of image data Img_α and executes a feature point matching process to align the feature points between the pieces of image data Img_α. The feature point matching process is performed using, for example, SfM (Structure from Motion) processing. Furthermore, the information processing device 100 combines the multiple pieces of image data Img_α through the feature point matching process to generate a road surface unevenness model that indicates information about the unevenness state of the road surface. The road surface unevenness model is generated using, for example, MVS (Multi-View Stereo) processing.

[0019] When there is no reference point within the measurement range D of the road surface to be used as a reference position when combining multiple image data, the road surface measurement system 1 of the present disclosure moves the measurement vehicle 10 to capture the images so that the entire capturing range of the multiple image data includes one or more out-of-range reference points set outside the measurement range D, and performs feature point matching processing using multiple image data obtained that each include at least one of the measurement range D, one or more out-of-range reference points, and a connection area set between the measurement range D and the one or more out-of-range reference points.

[0020] 2 to 4 are explanatory diagrams showing an outline of the processing performed by the road surface measurement system 1 according to the present disclosure. Figures 2 and 3 are explanatory diagrams showing out-of-range reference points M1 to M3 set outside the measurement range D inside a tunnel, the measurement range D, and connection areas X1 to X3. Figure 4 is an explanatory diagram showing the out-of-range reference points M1 to M3, the measurement range D, and connection areas X1 to X3 expanded in a plane.

[0021] For example, in a tunnel, it may not be possible to autonomously drive the measurement vehicle 10 using a global navigation satellite system (GNSS) such as a global positioning system (GPS). In this case, if a reference point that serves as a reference position when combining multiple image data exists within the measurement range D, it is possible to combine multiple image data of photographs of the road surface using the reference point as a constraint condition, and to generate an accurate road surface unevenness model that corresponds to the actual unevenness state of the road surface.

[0022] On the other hand, if there is no reference point within the road surface measurement range D that serves as a reference position when combining multiple image data pieces, even if the multiple image data pieces can be combined using feature point matching processing, it is not possible to identify which area of ​​the actual road the road surface unevenness model corresponds to. Therefore, in the road surface measurement system 1 according to the present disclosure, out-of-range reference points M1-M3 whose position information is managed are set outside the measurement range D, and images are taken of not only the out-of-range reference points M1-M3 but also an area including connection areas X1-X3 that connect the out-of-range reference points M1-M3 and the measurement range D, and the multiple image data obtained are combined using the out-of-range reference points M1-M3 as reference points.

[0023] The road surface measurement system 1 of the first embodiment acquires information on the position of one or more out-of-range reference points M1 to M3, and while moving the measurement vehicle 10, stops capturing images using an imaging device whose capturing range is a range that does not include any of the measurement range, one or more out-of-range reference points M1 to M3, and the connection areas X1 to X3, based on the position of the one or more out-of-range reference points M1 to M3, and then captures images using an imaging device whose capturing range is a range that includes at least one of the measurement range, one or more out-of-range reference points M1 to M3, and the connection areas X1 to X3, and then performs feature point matching processing using the multiple image data obtained.

[0024] The out-of-range reference points M1 to M3 are points that can be identified by their characteristic shape or appearance that exist in the vicinity of the measurement range D, and may be set on existing objects or may be installed during measurement. The position information of the out-of-range reference points M1 to M3 may be information that can identify their relative positions with respect to the measurement range D. For example, if the position coordinates of the measurement range D on the map data are known, the position information of the out-of-range reference points M1 to M3 may be information on their position coordinates on the map data. If the position coordinates of the measurement range D on the map data are not known, the position information of the out-of-range reference points M1 to M3 may be information on their relative positions with respect to the position of the measurement range D.

[0025] The measurement vehicle 10 and the information processing device 100 that constitute the road surface measurement system 1 according to this embodiment will be specifically described below.

[0026] <1-2. Measurement vehicle configuration example> First, a configuration example of the measurement vehicle 10 used in the road surface measurement system 1 according to this embodiment will be described.

[0027] (1-2-1. Overall structure) 5 and 6 are explanatory diagrams showing an example of the configuration of the measurement vehicle 10. FIG. 5 is a diagram of the measurement vehicle 10 as seen from the side, and FIG. 6 is a diagram of the measurement vehicle 10 as seen from above. In FIGS. 5 and 6, the basic forward direction of the measurement vehicle 10 is indicated as "front," and the backward direction is indicated as "rear." Hereinafter, the front-rear direction, left-right direction, and height direction of the measurement vehicle 10 refer to the front-rear direction, left-right direction, and height direction based on the "front" and "rear."

[0028] The measurement vehicle 10 generates a plurality of image data Img_α by capturing images of the road surface as a measurement target, and transmits the generated image data Img_α to the information processing device 100. The measurement vehicle 10 according to this embodiment is configured to be able to move on the road surface unmanned (capable of autonomous driving). The measurement vehicle 10 includes a carriage 3, an imaging unit 5, a surrounding condition detection unit 7, and a control device 50.

[0029] The bogie 3 includes a frame 11, wheels 13 (13F, 13R), a driving force source 15, and a steering device 17. The wheels 13 (13F, 13R), the driving force source 15, and the steering device 17 correspond to a driving device that moves the position of the measurement vehicle 10. The frame 11 forms the skeleton of the bogie 3. The structure and overall shape of the frame 11 shown in Figures 5 and 6 are merely examples, and the configuration of the frame 11 is not particularly limited. The wheels 13 include left and right front wheels 13F and rear wheels 13R, respectively.

[0030] The driving force source 15 outputs a driving force to rotate the rear wheel 13R. A driving motor is a typical example of the driving force source 15, but there is no particular limitation thereto as long as it is capable of outputting power to rotate and drive the rear wheel 13R.

[0031] The steering device 17 adjusts the steering angle of the front wheels 13F. An example of the steering device 17 is a steering device configured to rotate a pinion gear using a motor and move a rack left and right by the rotation of the pinion gear, but the configuration of the steering device 17 is not particularly limited.

[0032] The specific configuration of the dolly 3 is not particularly limited as long as it can support the imaging unit 5 and the surrounding situation detection unit 7 and can travel while adjusting the direction of movement. For example, the number of wheels 13 on the dolly 3 does not need to be four, but may be three, or five or more. Furthermore, the wheels 13 that are rotationally driven by the power output from the driving force source 15 may be the front wheels 13F, or both the front wheels 13F and the rear wheels 13R may be rotationally driven. Furthermore, the driving force source 15 may be four drive motors that respectively drive the four wheels 13.

[0033] Furthermore, the wheels whose steering angles are adjusted by the steering device 17 may be the rear wheels 13R, or all of the front wheels 13F and rear wheels 13R. When four drive motors drive the four wheels 13, respectively, and the direction of travel of the measurement vehicle can be adjusted by controlling the output of the drive motors, the steering device can be omitted.

[0034] The carriage 3 also includes a wheel speed sensor 41. The wheel speed sensor 41 outputs a sensor signal corresponding to the rotation speed of the rear wheel 13R. The wheel speed sensor 41 may also output a sensor signal corresponding to the rotation speed of the front wheel 13F.

[0035] The imaging unit 5 includes a plurality of imaging devices 21a to 21f, 22a to 22b (hereinafter collectively referred to as imaging devices 21, 22 unless a distinction is particularly required), irradiation lamps 25a to 25f, 26a to 26b (hereinafter collectively referred to as irradiation lamps 25, 26 unless a distinction is particularly required), and a reflector 23, and is provided at the rear of the trolley 3.

[0036] The multiple imaging devices 21, 22 are equipped with imaging elements such as CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor), and generate image data Img_α (α=1, 2···M) (M is the number of imaging devices 21, 22) capturing images of the road surface.

[0037] The imaging devices 21a to 21f are arranged along the left-right direction of the dolly 3, with their imaging directions facing the road surface. The imaging devices 21a to 21f are installed with the same height and installation angle (direction of imaging). In the measurement vehicle 10 of this embodiment, the six imaging devices 21a to 21f are arranged at equal intervals along the left-right direction of the dolly 3. The imaging devices 21 are installed so that the imaging ranges of adjacent imaging devices 21 overlap.

[0038] However, the installation positions and installation angles of the multiple imaging devices 21 do not have to be uniform as long as it is possible to maintain the degree of overlap of the imaging ranges of adjacent imaging devices 21. For example, the imaging directions of the imaging devices on both the left and right sides may be oriented toward the center.

[0039] The imaging devices 21 capture images of the road surface at set time intervals and output the generated image data Img_α (α=1, 2...M) to the processing unit 51. The imaging time intervals are set in consideration of the set range of the moving speed of the measurement vehicle 10 so that the imaging ranges of the time-series image data Img_α (α=1, 2...M) of each imaging device 21 overlap with each other. In other words, the time-series image data Img_α (α=1, 2...M) acquired by the processing unit 51 overlap with each other vertically and horizontally. This allows the imaging devices 21 to capture images of the entire road surface to be measured without any omissions.

[0040] The number of imaging devices 21 is not limited to six. Furthermore, the imaging devices 21 do not have to be arranged at equal intervals. However, by arranging the imaging devices 21 at equal intervals, the relative positional relationship of image data generated by adjacent imaging devices 21 becomes the same, which makes it easier to perform calculations.

[0041] The imaging devices 22a to 22b are arranged on both left-right outer sides of the dolly 3, with the imaging direction facing the side of the dolly 3. In the measurement vehicle 10 of this embodiment, one imaging device 22a to 22b is arranged on each of the left and right outer sides of the dolly 3 in the left-right direction. The imaging devices 22a to 22b are installed so that the imaging ranges of the imaging devices 21a, 21f, which are arranged on both ends of the imaging devices 21a to 22f, overlap. Note that the number of imaging devices 22 installed with the imaging direction facing the side of the dolly 3 may be two or more.

[0042] The irradiation lamps 25 are light sources for illuminating the image capture ranges of the image capture devices 21 and 22. In the measurement vehicle 10 of this embodiment, light emitted from the irradiation lamps 25 is reflected by the reflector 23 to illuminate the image capture ranges. Note that the number of irradiation lamps 25 is not particularly limited as long as the brightness (illuminance or luminosity) of the road surface imaged by each image capture device 21 is made uniform. Furthermore, the means for illuminating the image capture range is not limited to the configuration example of this embodiment, and may have any configuration.

[0043] The surrounding condition detection unit 7 has at least one sensor capable of detecting the conditions around the measurement vehicle 10. In this embodiment, the surrounding condition detection unit 7 has distance measurement sensors 31a and 31b (hereinafter collectively referred to as distance measurement sensors 31 unless a distinction is particularly required) and camera sensors 33a and 33b (hereinafter collectively referred to as camera sensors 33 unless a distinction is particularly required). The distance measurement sensor 31 may be, for example, a LiDAR (Light Detection and Ranging) sensor, but may also be a radar sensor or an ultrasonic sensor. The camera sensor 33 has an imaging element such as a CCD or CMOS, and generates image data.

[0044] 5 and 6, the distance measuring sensors 31a and 31b are installed so that the centers of their measurement ranges face forward on the left and right sides of the measurement vehicle 10. The distance measuring sensors 31a and 31b are installed so that they can detect objects, such as objects, mainly on the left and right sides of the measurement vehicle 10. The camera sensors 33a and 33b are installed so that the centers of their imaging ranges face forward on the measurement vehicle 10. The camera sensors 33a and 33b are installed so that they can capture a wide image of the area in front of the measurement vehicle 10.

[0045] The control device 50 includes one or more processors having a function of automatically moving the measurement vehicle 10 and a function of controlling the imaging devices 21 and 22, and one or more storage devices communicably connected to the one or more processors. The control device 50 also includes a communication interface for communicating with the information processing device 100. The control device 50 is communicably connected to various sensors and electronic control devices provided on the measurement vehicle 10.

[0046] (1-2-2. Control device) Next, the control device 50 provided in the measurement vehicle 10 will be described in detail.

[0047] (Control device configuration) FIG. 7 is a block diagram showing an example of the configuration of the road surface measurement system 1. The control device 50 includes a processing unit 51, a storage unit 53, and a communication unit 57. The processing unit 51 includes one or more processors such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and various peripheral components. Part or all of the processing unit 51 may be configured with updatable firmware or the like, or may be a program module or the like executed by commands from the CPU or the like.

[0048] The processing unit 51 functions as a device that controls the measurement vehicle 10 by having one or more processors execute a computer program. The computer program is a computer program that causes the processor to execute the operations to be performed by the processing unit 51, which will be described later. The computer program executed by the processor may be recorded on a recording medium that functions as a storage unit (memory) 53 provided in the control device 50, or may be recorded on a recording medium built into the processing unit 51 or any recording medium that can be externally attached to the control device 50.

[0049] Recording media for recording computer programs may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, and Blu-ray (registered trademark), magneto-optical media such as floptical disks, memory elements such as RAMs and ROMs, flash memories such as USB memories and SSDs, and other media capable of storing programs.

[0050] The processing unit 51 is communicatively connected to the driving force source 15, steering device 17, imaging devices 21 and 22, irradiation lamp 25, distance measurement sensor 31, camera sensor 33, wheel speed sensor 41, input unit 81, and notification unit 83.

[0051] The input unit 81 is a part that accepts input operations by a user. The input unit 81 is configured to include at least one of a keyboard, a mouse, a touchpad, and a microphone, for example. However, the type of the input unit 81 is not particularly limited.

[0052] The notification unit 83 is a part that provides a predetermined notification to the user. The notification unit 83 is configured to include, for example, at least one of a display device and a speaker. However, the type of the notification unit 83 is not particularly limited.

[0053] The storage unit 53 is configured with one or more storage elements such as RAM or ROM connected to the processing unit 51 so as to be able to communicate with it. However, the type and number of storage units 53 are not particularly limited. The storage unit 53 stores information such as computer programs executed by the processing unit 51, various parameters used in arithmetic processing, detection data, and arithmetic results. A part of the storage unit 53 is used as a work area.

[0054] The communication unit 57 is an interface for the processing unit 51 to communicate with the information processing device 100. The communication unit 57 may be an interface for communicating with the information processing device 100 via mobile communication, for example, but the communication method with the information processing device 100 is not particularly limited.

[0055] The control device 50 also includes an acceleration sensor 43, an angular velocity sensor 45, a GNSS sensor 47, and a map data storage unit 55, which are communicatively connected to the processing unit 51. Note that some or all of the acceleration sensor 43, the angular velocity sensor 45, the GNSS sensor 47, and the map data storage unit 55 may be provided separately from the control device 50.

[0056] The acceleration sensor 43 and the angular velocity sensor 45 function as an inertial measurement unit. The acceleration sensor 43 outputs a sensor signal corresponding to the acceleration in each of the three axial directions along the longitudinal, lateral, and height directions of the measurement vehicle 10. The angular velocity sensor 45 outputs a sensor signal corresponding to the angular velocity around each of the three axes along the longitudinal, lateral, and height directions of the measurement vehicle 10. The acceleration sensor 43 and the angular velocity sensor 45 may be integrated sensors as an IMU (inertial measurement unit) sensor, or may be separately provided sensors.

[0057] The GNSS sensor 47 receives satellite signals from positioning satellites such as GPS satellites, and transmits position information of the measurement vehicle 10 contained in the satellite signals to the processing unit 51. The position information may be information on latitude and longitude. Note that the GNSS sensor 47 may be configured to be able to receive satellite signals from satellite systems other than GPS satellites.

[0058] The map data storage unit 55 is configured by a storage element such as RAM or ROM communicably connected to the processing unit 51, or a storage medium such as an HDD, CD, DVD, SSD, USB flash, or storage device. The map data stored in the map data storage unit 55 is associated with latitude and longitude information, and the processing unit 51 can identify the position of the measurement vehicle 10 on the map data based on the latitude and longitude information of the measurement vehicle 10 transmitted from the GNSS sensor 47.

[0059] (Configuration of processing unit) Next, the configuration of the processing unit 51 of the control device 50 will be described.

[0060] The processing unit 51 starts capturing images of the road surface using the imaging devices 21 and 22, and executes processing to transmit the image data to the information processing device 100. The processing unit 51 includes an acquisition unit 61, an imaging condition setting unit 63, an autonomous driving control unit 67, a self-position estimation unit 65, an imaging device driving unit 69, and a communication control unit 71. Each of these units is a function realized by execution of a computer program by one or more processors. However, some or all of the acquisition unit 61, the imaging condition setting unit 63, the autonomous driving control unit 67, the self-position estimation unit 65, the imaging device driving unit 69, and the communication control unit 71 may be configured using analog circuits.

[0061] The acquisition unit 61 acquires image data generated by the imaging devices 21 and 22. The acquisition unit 61 acquires image data generated by the imaging devices 21 and 22 at set time intervals. The acquisition unit 61 also acquires detection data or sensor signals (hereinafter collectively referred to as "detection information") output from the distance measurement sensor 31, camera sensor 33, wheel speed sensor 41, acceleration sensor 43, angular velocity sensor 45, and GNSS sensor 47. The acquisition unit 61 acquires the detection information output from each sensor at a predetermined calculation cycle.

[0062] The imaging condition setting unit 63 sets imaging conditions as operating conditions for the imaging devices 21, 22. Specifically, the imaging condition setting unit 63 sets the moving speed of the imaging devices 21, 22, i.e., the moving speed of the measurement vehicle 10. The imaging condition setting unit 63 sets the moving speed so that the imaging ranges of the image data acquired in time series partially overlap, based on the length of the road surface corresponding to the pixel size of the image data captured by each of the imaging devices 21, 22 and the time interval between images captured by the imaging devices 21, 22. The imaging condition setting unit 63 also sets the aperture (F-number), shutter speed, and ISO sensitivity of the imaging devices 21, 22.

[0063] Information for setting the imaging conditions may be specified by the user. The user may manually set some or all of the acquired imaging conditions. The imaging condition setting unit 63 may also set the output (illuminance or brightness) of the irradiation lamp 25. Note that the imaging condition setting unit 63 does not need to perform setting processing for conditions that are fixed regardless of other conditions.

[0064] The self-position estimation unit 65 estimates the position of the measurement vehicle 10 on a predetermined three-dimensional coordinate system (hereinafter also referred to as "self-position"). When the self-position estimation unit 65 cannot acquire position information of the measurement vehicle 10 from the GNSS sensor 47, it estimates the self-position of the measurement vehicle 10 by a predetermined calculation. The self-position of the measurement vehicle 10 indicates the position of the measurement vehicle 10 on a predetermined three-dimensional coordinate system with the measurement start position as the origin, for example. The self-position of the measurement vehicle 10 may be a predetermined reference position (x0, y0, z0) arbitrarily set within the measurement vehicle 10, or may be, for example, the position of the center of gravity of the measurement vehicle 10, or may be the position of any one of the multiple image capturing devices 21a to 21f.

[0065] When predetermined guide information is detected based on the measurement data of the distance measurement sensor 31, the self-position estimation unit 65 estimates the self-position of the measurement vehicle 10 based on the guide information and the detection information of the wheel speed sensor 41, the acceleration sensor 43, and the angular velocity sensor 45. The predetermined guide information is, for example, a three-dimensional object that does not have speed and is installed with a predetermined or greater continuity along the road, such as a side wall, a curb, or a guardrail, and is an object that can serve as a reference when moving the measurement vehicle 10.

[0066] Furthermore, in this embodiment, if the self-position estimation unit 65 cannot detect guide information based on measurement data from the distance measurement sensor 31, and if the self-position estimation unit 65 can detect guide information based on image data generated by the camera sensor 33, it estimates the self-position of the measurement vehicle 10 based on the guide information and the detection information from the wheel speed sensor 41, acceleration sensor 43, and angular velocity sensor 45. On the other hand, if the self-position estimation unit 65 cannot detect guide information by either the distance measurement sensor 31 or the camera sensor 33, it estimates the self-position of the measurement vehicle 10 based on the detection information from the wheel speed sensor 41, acceleration sensor 43, and angular velocity sensor 45.

[0067] The autonomous driving control unit 67 controls the autonomous driving of the measurement vehicle 10. Autonomous driving refers to a driving state in which a processor or the like controls the driving of the driving force source 15 and the steering device 17 based on detection information, causing the measurement vehicle 10 to move along a predetermined target route.

[0068] In this embodiment, the autonomous driving control unit 67 basically moves the measurement vehicle 10 along the measurement route based on the position information of the measurement vehicle 10 detected by the GNSS sensor 47 and information about the measurement route set on map data. As long as the GNSS sensor 47 is in an area where it can receive satellite signals, the autonomous driving control unit 67 can move the measurement vehicle 10 along the preset measurement route using the position information detected by the GNSS sensor 47.

[0069] However, in areas where satellite signals do not reach or are difficult to reach, such as inside tunnels, the position information detected by the GNSS sensor 47 cannot be used. For this reason, in this embodiment, when road surface measurement is performed in such an area, the autonomous driving control unit 67 moves the measurement vehicle 10 using information about the self-position of the measurement vehicle 10 estimated by the self-position estimation unit 65. For example, the autonomous driving control unit 67 executes processing to move the measurement vehicle 10 along the measurement route while maintaining the distance between the guide information and the measurement vehicle 10 at a predetermined set distance.

[0070] The imaging device driving unit 69 controls the driving of the imaging devices 21 and 22. The imaging device driving unit 69 controls the imaging by the imaging devices 21 and 22 in accordance with the imaging conditions (aperture (F value), shutter speed, and ISO sensitivity) set by the imaging condition setting unit 63.

[0071] The communication control unit 71 controls the transmission and reception of information or data to and from the information processing device 100 .

[0072] (1-2-3. Processing operation of the processing unit) Next, the processing operation by the processing unit 51 of the control device 50 will be described.

[0073] 8 is a flowchart of the processing operation of the processing unit 51. The flowchart described below is a flowchart executed when the position information of the GNSS sensor 47 cannot be acquired, for example, because the GNSS sensor 47 cannot receive satellite signals.

[0074] First, the autonomous driving control unit 67 acquires information on a measurement range for measuring the unevenness of the road surface (measurement range information) (step S11). The measurement range information may be set by an operator or may be input from the information processing device 100.

[0075] Next, the imaging device driving unit 69 acquires information about out-of-range reference points M1 to M3 that are set outside the measurement range D (step S13). The information about the out-of-range reference points M1 to M3 includes information about the position of each of the out-of-range reference points M1 to M3. For example, if the position coordinates of the measurement range D on the map data are known, the information about the position of the out-of-range reference points M1 to M3 may be information about the position coordinates on the map data. If the position coordinates of the measurement range D on the map data are not known, the information about the position of the out-of-range reference points M1 to M3 may be information about their relative positions with respect to the position of the measurement range D.

[0076] Next, the autonomous driving control unit 67 starts the autonomous driving of the measurement vehicle 10 (step S15). If the autonomous driving control unit 67 cannot use the position information detected by the GNSS sensor 47, it moves the measurement vehicle 10 using at least one of information on the relative position of the measurement vehicle 10 with respect to the measurement range and guide information present in the periphery of the measurement range, and information on the self-position of the measurement vehicle 10 estimated by the self-position estimation unit 65.

[0077] FIG. 9 is a flowchart showing the autonomous driving control process of the measurement vehicle 10 performed by the autonomous driving control unit 67. The autonomous driving control unit 67 acquires information about the measurement route along which the measurement vehicle 10 will travel within the measurement range (step S31). The information about the measurement route may be set in advance by the user, or may be set by the autonomous driving control unit 67 based on the specified measurement range D. The measurement route includes information about at least the measurement start position and the measurement end position. For example, the self-position estimation unit 65 records the position coordinates of the reference position (x0, y0, z0) of the measurement vehicle 10 at the start of the measurement process as the start position. The recorded reference position does not need to be specified by specific latitude and longitude information, and may be recorded as an arbitrary point in arbitrary three-dimensional coordinates (for example, the origin of a three-dimensional coordinate system).

[0078] Next, the autonomous driving control unit 67 acquires information on the current position (x, y) and moving direction of the measurement vehicle 10 (step S33). The current position is calculated based on the position information detected by the GNSS sensor 47 or the odometry information estimated by the self-position estimation unit 65.

[0079] FIG. 10 is a flowchart showing an example of the self-position estimation process performed by the self-position estimation unit 65. The self-position estimation unit 65 acquires the distance from the measurement vehicle 10 to the guide information based on the detection information of the distance measurement sensor 31 (step S41). Next, the self-position estimation unit 65 acquires acceleration and angular velocity information obtained from the detection information of the acceleration sensor 43 and the angular velocity sensor 45 acquired in the current calculation cycle (step S43).

[0080] Next, the self-position estimation unit 65 calculates odometry information including the self-position of the measurement vehicle 10 based on information on the distance from the measurement vehicle 10 to the guide information, information on the acceleration and angular velocity of the measurement vehicle 10, and information on the position of the measurement vehicle 10 calculated in the previous calculation cycle (step S45). For example, the self-position estimation unit 65 calculates the position of the measurement vehicle 10 on a three-dimensional coordinate system with the start position recorded in step S31 as the origin.

[0081] Next, the self-position estimation unit 65 records data of the position (position coordinates of the self-position) P_n(x_n, y_n) of the measurement vehicle 10 calculated in the current calculation cycle (t_n) in the storage unit 53 (step S47). In this way, the self-position estimation unit 65 can estimate the position of the measurement vehicle 10 within the measurement range even when the position information detected by the GNSS sensor 47 cannot be used.

[0082] 9, the autonomous traveling control unit 67 sets the drive amount of the driving force source 15 and the steering device 17 (step S35). For example, the autonomous traveling control unit 67 sets the drive amount of the driving force source 15 so that the travel speed of the measurement vehicle 10 calculated based on the detection information of the wheel speed sensor 41 becomes the travel speed set by the imaging condition setting unit 63. In addition, the autonomous traveling control unit 67 sets the angular velocity of the change in the steering angle (steering angular velocity) based on information on the current travel direction of the measurement vehicle 10 and information on the measurement route, and sets the drive amount of the steering device 17 according to the steering angular velocity.

[0083] Next, the autonomous driving control unit 67 controls the driving of the driving force source 15 and the steering device 17 in accordance with the driving amount set in step S35 (step S37). This allows the measurement vehicle 10 to autonomously drive along the measurement route.

[0084] Returning to FIG. 8, while the autonomous driving control unit 67 is causing the measurement vehicle 10 to autonomously drive, the imaging device driving unit 69 determines whether the imaging range of each of the imaging devices 21 and 22 includes any of the measurement range D, the out-of-range reference points M1 to M3, and the connection areas X1 to X3 (step S17). Based on the self-position of the measurement vehicle 10 traveling within the measurement range D and the information on the positions of the out-of-range reference points M1 to M3, the imaging device driving unit 69 determines whether the imaging range of each of the imaging devices 21 and 22 includes any of the measurement range D, the out-of-range reference points M1 to M3, and the connection areas X1 to X3. The connection areas X1 to X3 are defined, for example, as areas within a predetermined distance on both sides of the shortest straight line connecting each of the out-of-range reference points M1 to M3 and the measurement range D. The predetermined distance can be, for example, 1 to 3 m. However, the method for setting the connection areas X1 to X3 is not limited to the above example.

[0085] If the imaging device driving unit 69 determines that the imaging range of each of the imaging devices 21, 22 includes any of the measurement range D, out-of-range reference points M1-M3, and connection regions X1-X3 (S17 / Yes), it performs imaging using all of the imaging devices 21, 22 (step S19). On the other hand, if the imaging device driving unit 69 does not determine that the imaging range of each of the imaging devices 21, 22 includes any of the measurement range D, out-of-range reference points M1-M3, and connection regions X1-X3 (S17 / No), it performs imaging except for imaging devices whose imaging range does not include any of the measurement range D, out-of-range reference points M1-M3, and connection regions X1-X3 (step S21). In this case, the generated image data includes at least any of the measurement range D, out-of-range reference points M1-M3, and connection regions X1-X3.

[0086] Next, the imaging device driving unit 69 stores the generated image data Img_α in the storage unit 53 together with information on the numbers α (α=1, 2, . . . M) of the imaging devices 21 and 22 (step S23).

[0087] Next, the communication control unit 71 transmits the stored image data Img_α to the information processing device 100 (step S25). While the autonomous driving and photographing processes are being executed, the autonomous driving control unit 67 determines whether or not traveling along the measurement route has been completed (step S27). For example, the autonomous driving control unit 67 determines that traveling along the measurement route has been completed when the measurement vehicle 10 reaches the measurement end position. If the autonomous driving control unit 67 does not determine that traveling along the measurement route has been completed (S27 / No), the process returns to step S17, and the processing unit 51 continues executing the autonomous driving and photographing processes. On the other hand, if the autonomous driving control unit 67 determines that traveling along the measurement route has been completed (S27 / Yes), the process ends.

[0088] FIG. 11 is an explanatory diagram showing the imaging range of image data transmitted from the measurement vehicle 10 to the information processing device 100 in this embodiment, and shows image data generated while the measurement vehicle 10 is autonomously traveling near the out-of-range reference point M1 in the area D1 shown in FIG. 1. Note that image data generated by the imaging device 22b is omitted from FIG. 11. The imaging devices 21a to 21f, whose imaging direction is directed toward the road surface in the area D1, capture images every cycle. On the other hand, the imaging device 22a, whose imaging direction is directed to the side of the measurement range D, captures images only in cycles whose imaging range includes the out-of-range reference point M1 or the connection area X1.

[0089] As described above, in this embodiment, the control device 50 of the measurement vehicle 10 transmits image data generated by photographing using the imaging devices 21, 22, whose photographing range includes at least one of the measurement range D, out-of-range reference points M1 to M3, and connection areas X1 to X3, to the information processing device 100.

[0090] <1-3. Information processing devices> Next, the information processing device 100 will be described in detail.

[0091] (1-3-1. Configuration of information processing device) 7, the information processing device 100 includes a communication unit 101, a data processing unit 103, and a storage unit 105. The communication unit 101 is an interface for the data processing unit 103 to communicate with the control device 50. The communication unit 101 may be an interface for communicating with the control device 50 via mobile communication, for example, but the communication method with the control device 50 is not particularly limited.

[0092] The data processing unit 103 is configured with one or more processors such as a CPU or a GPU, and various peripheral components. Part or all of the processing unit 51 may be configured with updatable components such as firmware, or may be a program module executed by instructions from the CPU, etc.

[0093] The data processing unit 103 functions as a device that realizes the functions described below by one or more processors executing a computer program. The computer program is a computer program that causes the processor to execute the operations described below that should be performed by the data processing unit 103. The computer program executed by the processor may be recorded on a recording medium that functions as a storage unit (memory) 105 provided in the information processing device 100, or may be recorded on a recording medium built into the data processing unit 103 or any recording medium that can be externally attached to the information processing device 100.

[0094] Recording media for recording computer programs may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, and Blu-ray (registered trademark), magneto-optical media such as floptical disks, memory elements such as RAMs and ROMs, flash memories such as USB memories and SSDs, and other media capable of storing programs.

[0095] The storage unit 105 is configured by one or more storage elements such as RAM or ROM, or storage media such as HDD or SSD, communicably connected to the data processing unit 103. However, the type and number of storage units 105 are not particularly limited. The storage unit 105 stores information such as computer programs executed by the data processing unit 103, various parameters used in arithmetic processing, detection data, and arithmetic results. A part of the storage unit 105 is used as a work area.

[0096] The data processing unit 103 receives the image data Img_α, extracts feature points from the received image data Img_α (α=1, 2...M), matches the feature points using SfM processing, and aligns the image data Img_α (α=1, 2...M). At that time, the image data Img_α (α=1, 2...M) is synthesized using out-of-range reference points M1 to M3 in the image data Img_α (α=1, 2...M) as constraint conditions. The data processing unit 103 also generates a three-dimensional road surface unevenness model that reflects data indicating the unevenness of the road surface that appears as feature points in the aligned image data Img_α (α=1, 2...M).

[0097] 12 shows a flowchart of the model generation process by the data processing unit 103. Note that the flowchart shown below shows an example in which the control device 50 sequentially transmits image data Img_α captured by the imaging device 21 to the information processing device 100 at predetermined calculation intervals.

[0098] The data processing unit 103 acquires image data Img_α_n (α=1, 2...M) transmitted from the control device 50 (step S51). The measurement vehicle 10 used in the road surface measurement system 1 of this embodiment is equipped with a total of eight imaging devices 21a-21f, 22a-22b, and the data processing unit 103 acquires image data Img_α_n (α=1, 2...8) received from the measurement vehicle 10 in the current calculation cycle (t_n). The data processing unit 103 stores the acquired image data Img_α_n (α=1, 2...8) in the memory unit 105.

[0099] Next, the data processing unit 103 extracts out-of-range reference points M1 to M3 and feature points from the acquired image data Img_α_n (α=1, 2, M) (step S53). Feature points may be points where the amount of change in luminance (feature amount) between adjacent pixels in the image data Img_α_n is equal to or greater than a predetermined threshold, but other feature amounts may also be used. The out-of-range reference points M1 to M3 can be identified as one of a group of extracted feature points whose pattern shape matches the pattern shape used to identify the out-of-range reference points M1 to M3 in advance. A method for extracting feature points from image data may be a conventionally known method, and therefore a detailed description thereof will be omitted.

[0100] Next, the data processing unit 103 executes a process of matching feature points between image data Img_α (α=1, 2...M) using the extracted out-of-range reference points M1 to M3 as constraint conditions (step S55). This matching process includes not only matching feature points between image data Img_α_n (α=1, 2...M) acquired in the current calculation cycle (t_n), but also matching feature points between image data Img_α (α=1, 2...M) acquired in calculation cycles previous to the previous one and stored in the storage unit 105. The process of matching feature points between image data may be executed using a conventionally known method, and therefore a detailed description thereof will be omitted.

[0101] Next, the data processing unit 103 constructs three-dimensional point cloud data restored by the principle of triangulation using the image data Img_α (α=1, 2...M) aligned by the feature point matching process, and generates a road surface unevenness model (step S57). In this embodiment, the data processing unit 103 constructs three-dimensional point cloud data by MVS (Multi-View Stereo) processing, and generates a road surface unevenness model. However, the method for generating a road surface unevenness model composed of three-dimensional point cloud data is not particularly limited.

[0102] Next, the data processing unit 103 determines whether or not to end the model generation process (step S59). For example, the data processing unit 103 may determine to end the model generation process when a predetermined time has elapsed since the control device 50 stopped transmitting image data Img_α. Alternatively, the data processing unit 103 may determine to end the model generation process when the control device 50 transmits a signal indicating the end of measurement.

[0103] If the data processing unit 103 does not determine that the model generation process should be terminated (S59 / No), the data processing unit 103 returns to step S51 and continues the model generation process. On the other hand, if the data processing unit 103 determines that the model generation process should be terminated (S59 / Yes), the data processing unit 103 terminates the model generation process.

[0104] As described above, in this embodiment, the information processing device 100 acquires only image data generated by capturing images using the imaging devices 21, 22 whose capturing ranges include at least one of the measurement range D, the out-of-range reference points M1 to M3, and the connection areas X1 to X3 from the measurement vehicle 10. Furthermore, the information processing device 100 performs a matching process on the acquired image data using the out-of-range reference points M1 to M3 as constraint conditions, and generates a road surface unevenness model.

[0105] <1-4. Effects> As described above, when there is no reference point within the measurement range D that serves as a reference position when combining multiple image data, the road surface measurement system 1 according to this embodiment moves the measurement vehicle 10 to capture images so that the entire capturing range of the multiple image data includes one or more out-of-range reference points M1-M3 that are set outside the measurement range. Furthermore, the road surface measurement system 1 performs feature point matching processing using multiple image data that includes at least one of the measurement range D, one or more out-of-range reference points M1-M3, and connection regions X1-X3 that are set between the measurement range D and one or more out-of-range reference points M1-M3, among the obtained image data.

[0106] As a result, even if position information cannot be obtained from the GNSS sensor and there are no reference points within the measurement range D, it is possible to generate a road surface unevenness model by synthesizing image data using the out-of-range reference points M1 to M3. Furthermore, because the image data used in the matching process each includes at least one of the measurement range D, one or more out-of-range reference points M1 to M3, and the connection regions X1 to X3, an increase in the time required to generate the road surface unevenness model is suppressed, and the road surface unevenness model can be generated efficiently.

[0107] Furthermore, the road surface measurement system 1 of this embodiment is configured to acquire information on the positions of one or more out-of-range reference points M1 to M3, and while moving the measurement vehicle 10, stop capturing images using an imaging device whose capturing range is a range that does not include any of the measurement range D, one or more out-of-range reference points M1 to M3, and the connection areas X1 to X3, based on the positions of the one or more out-of-range reference points M1 to M3, capture images using an imaging device whose capturing range is a range that includes at least one of the measurement range D, one or more out-of-range reference points M1 to M3, and the connection areas X1 to X3, and perform feature point matching processing using the multiple image data obtained.

[0108] This reduces the number of image data transmitted from the measurement vehicle 10 to the information processing device 100. Furthermore, the information processing device 100 can perform matching processing without selecting and discarding the received image data.

[0109] The road surface measurement system 1 according to this embodiment includes a distance measurement sensor and an inertial measurement unit, and estimates the vehicle's own position, which is the movement history of the vehicle, based on the measurement results of the inertial measurement unit while the vehicle is moving. Based on the estimated position information and the position information of the out-of-range reference points M1-M3 relative to the vehicle's own position detected by the distance measurement sensor, the system stops capturing images using an imaging device whose capturing range does not include the measurement range D, one or more out-of-range reference points M1-M3, or the connection areas X1-X3, and starts capturing images using an imaging device whose capturing range includes at least one of the measurement range D, one or more out-of-range reference points M1-M3, and the connection areas X1-X3. This improves the accuracy of determining whether the capturing range includes at least one of the measurement range D, one or more out-of-range reference points M1-M3, and the connection areas X1-X3, and prevents an increase in the amount of image data.

[0110] <<2. Second Embodiment>> A road surface measurement system according to a second embodiment of the present disclosure is configured to capture images while moving a measurement vehicle, transmit all of the obtained multiple image data to an information processing device, and have the information processing device exclude image data that does not include the measurement range, one or more out-of-range control points, or connection areas, and perform feature point matching processing using multiple image data that include at least one of the measurement range, one or more out-of-range control points, and connection areas.Unlike the road surface measurement system according to the first embodiment, the road surface measurement system according to the second embodiment transmits all image data from the measurement vehicle to the information processing device, and the information processing device selects and discards the image data to perform matching processing.

[0111] 13 is a flowchart showing a main routine of the processing operation by the processing unit 51 of the control device 50 of the road surface measurement system according to the second embodiment. In Fig. 13, steps showing the same processes as those in the flowchart shown in Fig. 8 are denoted by the same reference numerals.

[0112] As in the first embodiment, the autonomous driving control unit 67 acquires measurement range information (step S11), and then the imaging device driving unit 69 acquires information on out-of-range reference points M1 to M3 set outside the measurement range D (step S13). Next, the autonomous driving control unit 67 starts the autonomous driving of the measurement vehicle 10 (step S15), and the imaging device driving unit 69 captures images using all of the imaging devices 21 and 22 (step S19). In this embodiment, the imaging device driving unit 69 captures images using all of the imaging devices 21 and 22 at each cycle and generates image data, regardless of whether the capturing range includes any of the measurement range, one or more out-of-range reference points, and the connection area.

[0113] Next, the imaging device driving unit 69 stores the generated image data Img_α in the storage unit 53 together with information on the number α (α=1, 2...M) of the imaging devices 21, 22 (step S23). Next, the communication control unit 71 transmits the stored image data Img_α to the information processing device 100 (step S25). While the autonomous traveling and photographing process are being executed, the autonomous traveling control unit 67 determines whether traveling along the measurement route has been completed (step S27). If the autonomous traveling control unit 67 does not determine that traveling along the measurement route has been completed (S27 / No), the process returns to step S17, and the processing unit 51 continues executing the autonomous traveling and photographing process. On the other hand, if the autonomous traveling control unit 67 determines that traveling along the measurement route has been completed (S27 / Yes), the process ends.

[0114] 14 is an explanatory diagram showing the imaging range of image data transmitted from the measurement vehicle 10 to the information processing device 100 in this embodiment, and shows image data generated while the measurement vehicle 10 is autonomously traveling near the out-of-range reference point M1 in the area D1 shown in FIG. 1. Note that image data generated by the imaging device 22b is omitted from FIG. 14. Not only the imaging devices 21a to 21f whose imaging direction is directed toward the road surface in area D1, but also the imaging device 22a whose imaging direction is directed to the side of the measurement range D similarly performs imaging every cycle.

[0115] As described above, in this embodiment, the control device 50 of the measurement vehicle 10 transmits image data captured and generated by all of the imaging devices 21, 22 to the information processing device 100, regardless of whether the shooting range includes any of the measurement range D, out-of-range reference points M1 to M3, and connection areas X1 to X3.

[0116] 15 is a flowchart showing a processing routine of the data processing unit 103 of the information processing device 100 of the road surface measurement system according to the second embodiment. In Fig. 15, steps showing the same processes as those in the flowchart shown in Fig. 12 are denoted by the same reference numerals.

[0117] The data processing unit 103 acquires image data Img_α_n (α=1, 2...M) transmitted from the control device 50 (step S51). Next, the data processing unit 103 determines whether or not each image data Img_α_n (α=1, 2...M) includes the measurement range D (step S61). For example, the data processing unit 103 determines whether or not the image data includes the measurement range D based on the position of the measurement vehicle 10 and the positions of the imaging devices 21 and 22 when each image data was captured. If the data processing unit 103 determines that each image data Img_α_n (α=1, 2...M) includes the measurement range D (S61 / Yes), the process proceeds to step S53.

[0118] On the other hand, if the data processing unit 103 does not determine that each image data Img_α_n (α=1, 2...M) includes the measurement range D (S61 / No), it determines whether or not each image data Img_α_n (α=1, 2...M) includes out-of-range reference points M1 to M3 (step S63). For example, the data processing unit 103 determines whether or not the image data includes out-of-range reference points M1 to M3 based on the position of the measurement vehicle 10, the positions of the imaging devices 21 and 22, and the positions of the out-of-range reference points M1 to M3 when each image data was captured. If the data processing unit 103 determines that each image data Img_α_n (α=1, 2...M) includes out-of-range reference points M1 to M3 (S63 / Yes), it proceeds to step S53.

[0119] On the other hand, if the data processing unit 103 does not determine that each image data Img_α_n (α=1, 2...M) includes out-of-range reference points M1 to M3 (S63 / No), it determines whether each image data Img_α_n (α=1, 2...M) includes connection regions X1 to X3 (step S65). For example, the data processing unit 103 determines whether the image data includes connection regions X1 to X3 based on the position of the measurement vehicle 10, the positions of the imaging devices 21 and 22, and the positions of the out-of-range reference points M1 to M3 when each image data was captured.

[0120] If the data processing unit 103 determines that each image data Img_α_n (α=1, 2...M) includes the connected regions X1 to X3 (S65 / Yes), the process proceeds to step S53. On the other hand, if the data processing unit 103 does not determine that each image data Img_α_n (α=1, 2...M) includes the connected regions X1 to X3 (S65 / No), the data processing unit 103 excludes image data that does not include any of the measurement range D, out-of-range reference points M1 to M3, and connected regions X1 to X3 from the matching targets (step S67), and the process proceeds to step S53.

[0121] After step S53, the data processing unit 103 extracts out-of-range reference points M1 to M3 and feature points from image data Img_α_n (α=1, 2...M) including any of the measurement range D, out-of-range reference points M1 to M3, and connection regions X1 to X3, as in the first embodiment (step S53). Next, the data processing unit 103 executes a process of matching feature points between image data Img_α (α=1, 2...M) using the extracted out-of-range reference points M1 to M3 as constraint conditions (step S55). Furthermore, the data processing unit 103 constructs three-dimensional point cloud data restored by the principle of triangulation using the image data Img_α (α=1, 2...M) aligned by the feature point matching process, and generates a road surface unevenness model (step S57).

[0122] Next, the data processing unit 103 determines whether or not to end the model generation process (step S59). If the data processing unit 103 does not determine to end the model generation process (S59 / No), it returns to step S51 and continues the model generation process. On the other hand, if the data processing unit 103 determines to end the model generation process (S59 / Yes), it ends the model generation process.

[0123] 16 is an explanatory diagram showing the image data synthesis process performed by the information processing device 100 in this embodiment, and shows the process of synthesizing image data generated while the measurement vehicle 10 is autonomously traveling near the out-of-range reference point M1 in the area D1 shown in FIG. 1. Note that image data generated by the imaging device 22b is omitted from FIG. 16. In this embodiment, the information processing device 100 excludes from the matching target, among the image data acquired from the measurement vehicle 10, image data that does not include the measurement range D, the out-of-range reference points M1 to M3, or the connection area X1 to X3, and performs matching process on the image data that includes the measurement range D, the out-of-range reference point M1, or the connection area X1, using the out-of-range reference points M1 to M3 as constraint conditions, to generate a road surface unevenness model.

[0124] As described above, the road surface measurement system 1 according to this embodiment is configured to take images using the imaging devices 21 and 22 while moving the measurement vehicle 10, exclude from the multiple image data obtained image data that does not include the measurement range D, one or more out-of-range reference points M1 to M3, and connection areas X1 to X3, and then perform feature point matching processing using multiple image data that includes at least one of the measurement range D, one or more out-of-range reference points M1 to M3, and connection areas X1 to X3.

[0125] As a result, even if position information cannot be obtained from the GNSS sensor and there are no reference points within the measurement range D, it is possible to generate a road surface unevenness model by synthesizing image data using the out-of-range reference points M1 to M3. Furthermore, because the image data used in the matching process each includes at least one of the measurement range D, one or more out-of-range reference points M1 to M3, and the connection regions X1 to X3, an increase in the time required to generate the road surface unevenness model is suppressed, and the road surface unevenness model can be generated efficiently.

[0126] <<3. Third Embodiment>> A road surface measurement system according to a third embodiment of the present disclosure is configured to capture images while moving a measurement vehicle, trim the multiple image data obtained to exclude areas that do not include the measurement range, one or more out-of-range control points, or connection areas, and then perform feature point matching processing using the multiple image data.Unlike the road surface measurement systems according to the first and second embodiments, the road surface measurement system according to the third embodiment further limits the range of image data for which feature point matching is performed in an information processing device and performs matching processing.

[0127] In this embodiment, the processing executed by the control device 50 of the measurement vehicle 10 may be processing according to either the first embodiment or the second embodiment. That is, the control device 50 of the measurement vehicle 10 may transmit image data generated by capturing images using the imaging devices 21 and 22 whose capturing ranges include at least one of the measurement range D, the out-of-range reference points M1 to M3, and the connection areas X1 to X3 to the information processing device 100. Alternatively, the control device 50 of the measurement vehicle 10 may transmit image data generated by capturing images using all of the imaging devices 21 and 22 to the information processing device 100, regardless of whether the capturing ranges include any of the measurement range D, the out-of-range reference points M1 to M3, and the connection areas X1 to X3.

[0128] 17 is a flowchart showing a processing routine of the data processing unit 103 of the information processing device 100 of the road surface measurement system according to the third embodiment. In Fig. 17, steps showing the same processes as those in the flowchart shown in Fig. 12 are denoted by the same reference numerals.

[0129] The data processing unit 103 acquires image data Img_α_n (α=1, 2...M) transmitted from the control device 50 (step S51). Next, the data processing unit 103 trims and deletes from the acquired image data Img_α_n (α=1, 2...M) an area that does not include any of the measurement range D, the out-of-range reference points M1 to M3, and the connection areas X1 to X3 (step S52). For example, the data processing unit 103 trims and deletes an area that does not include any of the measurement range D, the out-of-range reference points M1 to M3, and the connection areas X1 to X3 based on the position of the measurement vehicle 10, the positions of the imaging devices 21 and 22, and the positions of the out-of-range reference points M1 to M3 when each image data was captured.

[0130] After step S53, the data processing unit 103 extracts out-of-range reference points M1 to M3 and feature points from image data Img_α_n (α=1, 2...M) including any of the measurement range D, out-of-range reference points M1 to M3, and connection regions X1 to X3, as in the first embodiment (step S53). Next, the data processing unit 103 executes a process of matching feature points between image data Img_α (α=1, 2...M) using the extracted out-of-range reference points M1 to M3 as constraint conditions (step S55). Furthermore, the data processing unit 103 constructs three-dimensional point cloud data restored by the principle of triangulation using the image data Img_α (α=1, 2...M) aligned by the feature point matching process, and generates a road surface unevenness model (step S57).

[0131] Next, the data processing unit 103 determines whether or not to end the model generation process (step S59). If the data processing unit 103 does not determine to end the model generation process (S59 / No), it returns to step S51 and continues the model generation process. On the other hand, if the data processing unit 103 determines to end the model generation process (S59 / Yes), it ends the model generation process.

[0132] 18 is an explanatory diagram showing the image data synthesis process performed by the information processing device 100 in this embodiment, and shows the process of synthesizing image data generated while the measurement vehicle 10 is autonomously traveling near the out-of-range reference point M1 in the area D1 shown in FIG. 1. Note that image data generated by the imaging device 22b is omitted from FIG. 18. In this embodiment, the information processing device 100 trims and deletes areas of the image data acquired from the measurement vehicle 10 that do not include the measurement range D, the out-of-range reference points M1 to M3, or the connection areas X1 to X3, and performs a matching process on the image data that includes the measurement range D, the out-of-range reference point M1, or the connection area X1, using the out-of-range reference points M1 to M3 as constraints, to generate a road surface unevenness model.

[0133] As described above, the road surface measurement system 1 according to this embodiment is configured to take images using the imaging devices 21 and 22 while moving the measurement vehicle 10, trim the multiple image data obtained to exclude areas that do not include the measurement range D, one or more out-of-range reference points M1 to M3, and the connection areas X1 to X3, and then perform feature point matching processing using the multiple image data.

[0134] As a result, even if position information cannot be obtained from the GNSS sensor and there are no reference points within the measurement range D, it is possible to generate a road surface unevenness model by synthesizing image data using the out-of-range reference points M1 to M3. Furthermore, the image data used in the matching process is trimmed to an area that includes at least one of the measurement range D, one or more out-of-range reference points M1 to M3, and the connection areas X1 to X3, which prevents an increase in the time required to generate the road surface unevenness model and allows the road surface unevenness model to be generated efficiently.

[0135] <<4. Fourth Embodiment>> A road surface measurement system according to a fourth embodiment of the present disclosure is configured to automatically determine whether or not a reference point exists within the measurement range, and if no reference point exists within the measurement range, to perform a synthesis process of multiple image data using an out-of-range reference point.

[0136] FIG. 19 is a flowchart showing a main routine of the processing operation by the processing unit 51 of the control device 50 of the road surface measuring system according to the fourth embodiment.

[0137] The autonomous driving control unit 67 acquires measurement range information (step S71) and starts the autonomous driving of the measurement vehicle 10 (step S73). Next, the imaging device driving unit 69 captures images using the imaging devices 21a to 21f, whose imaging range is the road surface of the measurement range D (step S75). In this embodiment, the imaging device driving unit 69 first captures images of only the road surface of the measurement range D and generates image data.

[0138] Next, the imaging device driving unit 69 stores the generated image data Img_α in the storage unit 53 together with information on the number α (α=1, 2...M) of the imaging device 21 (step S77). Next, the communication control unit 71 transmits the stored image data Img_α to the information processing device 100 (step S79). As a result, the information processing device 100 executes a feature point matching process on the transmitted image data and determines whether or not a reference point exists in the measurement range D.

[0139] Next, the imaging device driving unit 69 determines whether or not a reference point exists in the measurement range D (step S81). For example, the imaging device driving unit 69 acquires the determination result by the information processing device 100 and determines whether or not a reference point exists in the measurement range D. If the imaging device driving unit 69 determines that a reference point exists in the measurement range D (S81 / Yes), it adds the imaging devices 22a and 22b whose imaging range includes the out-of-range reference points M1 to M3 and the connection regions X1 to X3 as imaging devices that will capture images, and proceeds to the processing of any of the first to third embodiments (step S83).

[0140] At that time, the autonomous driving control unit 67 may move the measurement vehicle 10 to a position where it can photograph any of the out-of-range reference points M1 to M3 based on position information of the multiple out-of-range reference points M1 to M3, position information of the measurement range D, and position information of the measurement vehicle 10, and then start processing of any of the first to third embodiments.

[0141] On the other hand, if the imaging device driving unit 69 does not determine that a reference point exists within the measurement range D (S81 / No), the autonomous driving control unit 67 determines whether or not traveling along the measurement route has been completed (step S85). If the autonomous driving control unit 67 does not determine that traveling along the measurement route has been completed (S85 / No), the process returns to step S75, and the processing unit 51 continues executing the autonomous traveling and imaging process. On the other hand, if the autonomous driving control unit 67 determines that traveling along the measurement route has been completed (S85 / Yes), the series of processes ends.

[0142] FIG. 20 is a flowchart showing a routine of the processing operation by the data processing unit 103 of the information processing device 100 of the road surface measurement system according to the fourth embodiment.

[0143] The data processing unit 103 acquires image data Img_α_n (α=1, 2...M) of the measurement range D transmitted from the control device 50 (step S91). Next, the data processing unit 103 extracts feature points from the image data Img_α_n (α=1, 2...M) of the measurement range D (step S93). Next, the data processing unit 103 executes a process of matching the extracted feature points between the image data Img_α (α=1, 2...M) (step S95).

[0144] Next, the data processing unit 103 determines whether or not a reference point exists within the measurement range D (step S97). For example, the data processing unit 103 compares feature points extracted from multiple image data acquired over a predetermined number of calculation cycles with a preset pattern of reference points to determine whether or not a continuously detected reference point exists. If the data processing unit 103 determines that a reference point exists within the measurement range D (S97 / Yes), it constructs three-dimensional point cloud data restored according to the principle of triangulation using image data Img_α (α=1, 2...M) aligned by feature point matching processing, with the reference point as a constraint condition, and generates a road surface unevenness model (step S99).

[0145] Next, the data processing unit 103 determines whether or not to end the model generation process (step S101). If the data processing unit 103 does not determine to end the model generation process (S101 / No), it returns to step S91 and continues the model generation process. On the other hand, if the data processing unit 103 determines to end the model generation process (S101 / Yes), it ends the model generation process.

[0146] On the other hand, if the data processing unit 103 does not determine that a reference point exists within the measurement range D (S97 / No), it notifies the control device 50 of the measurement vehicle 10 that a reference point does not exist within the measurement range D (step S103). Next, the data processing unit 103 proceeds to the processing of any one of the first to third embodiments (step S105).

[0147] As described above, the road surface measurement system 1 according to this embodiment automatically determines whether or not a reference point exists within the measurement range D, and if no reference point exists, proceeds to processing using an out-of-range reference point. Therefore, if a reference point exists within the measurement range D, feature point matching processing can be performed without including the out-of-range reference point, reducing the total number of images captured and the processing load.

[0148] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art to which the present disclosure pertains can conceive of various modifications or alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0149] For example, the road surface measurement system of the above embodiment is configured by a measurement vehicle that mainly acquires image data and an information processing device that mainly executes data processing, which are connected so that they can communicate with each other. However, the technology of the present disclosure is not limited to the above example. Some of the functions of the data processing unit of the information processing device may be provided in the measurement vehicle. Alternatively, the information processing device may be mounted on the measurement vehicle, or all of the functions of the data processing unit of the information processing device may be provided in a control device of the measurement vehicle, and the road surface measurement system may be configured only by the measurement vehicle.

[0150] In addition, the technology of the present disclosure can also be realized as a measurement vehicle equipped with the control device described in the above embodiment, a road surface measurement processing method using the control device, an information processing method executed by the above information processing device, a computer program that causes a computer to function as at least one of the above control device and information processing device, and a non-temporary tangible recording medium on which the computer program is recorded. [Explanation of symbols]

[0151] 1: Road surface measurement system 3: Cart 10: Measurement vehicle 21-22: Imaging device 23: Reflector 25·26: Irradiation lamp 31: Distance sensor 33: Camera sensor 41: Wheel speed sensor 43: Acceleration sensor 45: Angular velocity sensor 47: GNSS sensor 50: Control device 51: Processing section 53: Storage section 55: Map data storage unit 57: Communications Department 61: Acquisition part 63: Imaging condition setting section 65: Self-position estimation part 67: Autonomous driving control unit 69: Imaging device drive unit 71: Communication control unit 100: Information processing device 101: Communications Department 103: Data processing section 105: Storage section

Claims

1. A road surface measurement system generates a road surface unevenness model by combining multiple image data obtained by capturing images of the road surface using a feature point matching process while a measurement vehicle equipped with an image capturing device is driven autonomously, one or more control devices; The one or more control devices If there is no reference point within the measurement range that is to be used as a reference position when combining the plurality of image data, the measurement vehicle is moved to capture the entire photographing range of the plurality of image data so that one or more out-of-range reference points set outside the measurement range are included, performing the feature point matching process using the plurality of image data, each of which includes at least one of a measurement range, the one or more out-of-range reference points, and a connection region set between the measurement range and the one or more out-of-range reference points, among the obtained image data; Road surface measurement system.

2. The one or more control devices Obtaining information about the position of the one or more out-of-range reference points; While moving the measurement vehicle, based on the positions of the one or more out-of-range reference points, stopping capturing an image by an imaging device whose capturing range is a range that does not include any of the measurement range, the one or more out-of-range reference points, and the connection area, and capturing an image by an imaging device whose capturing range is a range that includes at least one of the measurement range, the one or more out-of-range reference points, and the connection area; performing the feature point matching process using the obtained plurality of image data; The road surface measurement system according to claim 1 .

3. The one or more control devices Photographing is performed while the measurement vehicle is moving, excluding image data that does not include any of the measurement range, the one or more out-of-range reference points, and the connection region from the obtained plurality of image data; performing the feature point matching process using the plurality of image data including at least one of the measurement range, the one or more out-of-range reference points, and the connection region; The road surface measurement system according to claim 1 .

4. The one or more control devices Photographing is performed while the measurement vehicle is moving, performing trimming processing to exclude an area that does not include any of the measurement range, the one or more out-of-range reference points, and the connection area from the plurality of image data obtained; performing the feature point matching process using the plurality of image data; The road surface measurement system according to claim 1 .

5. The one or more control devices determining whether the reference point exists within the measurement range, and if the reference point does not exist, performing a process using the reference point outside the range; The road surface measurement system according to claim 1 .

Images