Imaging system and imaging method

Abstract

To provide an imaging system and an imaging method that can appropriately perform imaging even when picking up images of areas of an object that are different in imaging distance.SOLUTION: A second imaging area is closer than a first imaging area from imaging reference positions of a first imaging unit and a second imaging unit in a direction intersecting a movement direction. When an imaging width that is an imaging range in the movement direction when the first imaging unit picks up an image of the first imaging area in single imaging is defined as a first imaging width, and an imaging width that is an imaging range in the movement direction when the second imaging unit picks up an image of the second imaging area in single imaging is defined as a second imaging width, a first imaging parameter for determining the first imaging width in the first imaging unit, and a second imaging parameter for determining the second imaging width in the second imaging unit are different from each other, so that the second imaging width falls within a predetermined range of the first imaging width.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to an imaging system and an imaging method. [Background technology]

[0002] For example, Patent Document 1 describes a slope inspection method in which the condition of a slope is inspected based on visual observation, and when a deformation is found, the condition and position are recorded. In this slope inspection method, a vehicle traveling on a flat surface at the top or toe side of the slope divides the slope in the height direction and continuously photographs the slope with a substantially uniform resolution using multiple cameras with different focal lengths, and the photographing positions are recorded by a position measurement means mounted on the vehicle. In addition, three-dimensional measurement data of the slope is created by photogrammetry using the continuously photographed images and the photographing positions and attitudes. In addition, the three-dimensional measurement data is used to visually observe the condition of the slope based on a direct projection image that has been projected in a direction directly facing the slope. Then, when a deformation is found, its condition and position are identified. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6498488 Summary of the Invention [Problem to be solved by the invention]

[0004] However, according to the inventors' intensive research, the slope inspection method of Patent Document 1 has room for improvement in terms of capturing appropriate images even when capturing images of areas of the object at different imaging distances.

[0005] The present invention has been made based on the above-mentioned concerns, and aims to provide an imaging system and an imaging method that enable appropriate imaging even when imaging areas of an object with different imaging distances. [Means for solving the problem]

[0006] The imaging system of the present invention is an imaging system having a first imaging section and a second imaging section, the imaging system being installed on a moving device and continuously imaging an object while moving in a moving direction, the first imaging section images a first imaging area which is a part of the object, the second imaging section images a second imaging area which is a part of the object, the second imaging area is closer to the imaging reference position of the first imaging section and the second imaging section in a direction intersecting the moving direction than the first imaging area, and the first imaging section is If an imaging width that is an imaging range in the movement direction when imaging one imaging area in one imaging is defined as a first imaging width, and an imaging width that is an imaging range in the movement direction when the second imaging unit images the second imaging area in one imaging is defined as a second imaging width, a first imaging parameter for determining the first imaging width in the first imaging unit and a second imaging parameter for determining the second imaging width in the second imaging unit are different so that the second imaging width is within a predetermined range of the first imaging width. Effect of the Invention

[0007] According to the present invention, it is possible to provide an imaging system and an imaging method that enable appropriate imaging even when imaging areas of an object having different imaging distances. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of a state inspection system. [Diagram 2] FIG. 13 is a diagram showing an example of how a slope condition is inspected using a condition inspection system. [Diagram 3] FIG. 13 is a diagram showing an example of a state of a slope. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a data acquisition device. [Diagram 5] FIG. 2 is a diagram for explaining a captured image acquired by a mobile system. [Figure 6] 3A and 3B are explanatory diagrams of a captured image and a distance measurement image. [Figure 7] FIG. 2 is an explanatory diagram of a plurality of imaging regions. [Figure 8] 10 is a diagram illustrating a schematic diagram of a difference between an imaging distance of a first slope imaging region by a first camera and an imaging distance of a second slope imaging region by a second camera. FIG. [Figure 9] FIG. 2 is a conceptual diagram of imaging when the first camera and the second camera have the same specifications. [Figure 10] FIG. 2 is a diagram showing the relationship between the focal length, the size of the image sensor, and the imaging width, which is the premise of the present invention. [Figure 11] 3 is a conceptual diagram of image capture by a first camera and a second camera in the present embodiment. FIG. [Figure 12] FIG. 1 is a diagram illustrating an imaging system according to a first numerical example. [Figure 13] 1 is a diagram for visually understanding imaging parameters of an imaging unit (camera). [Figure 14] FIG. 11 is a diagram showing imaging parameters of the first camera and the second camera in Numerical Example 1. [Figure 15] 1 is a schematic diagram illustrating the structure of a line sensor (image sensor). [Figure 16] FIG. 11 is a diagram illustrating an imaging system according to a numerical example 3. [Figure 17] FIG. 13 is a diagram showing imaging parameters of the first camera and the second camera (the second X camera and the second Y camera) in Numerical Example 3. [Figure 18] FIG. 11 is a sequence diagram showing an example of a display process in the state inspection system. [Figure 19] FIG. 11 is an explanatory diagram of operations on a display screen of the state inspection system. [Figure 20] 11 is a flowchart illustrating an example of a process of an information processing method. [Figure 21] 19 is an example of a display screen after the processing shown in FIG. 18. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] <Definitions of terms, etc.> The imaging system and imaging method of the present embodiment will be described in detail below with reference to the drawings. In the following description, the X-axis, Y-axis, and Z-axis directions are based on the arrow directions shown in the drawings. The X-axis, Y-axis, and Z-axis directions define a three-dimensional space that is orthogonal to each other. The X-axis direction may be read as the moving direction (advancement direction, running direction) of a moving body (moving device). The Y-axis direction may be read as the height direction (vertical direction, perpendicular direction) of the slope of the object. The Z-axis direction may be read as the direction intersecting with the moving direction of the moving body (moving device) (depth direction from the moving body (moving device) toward the slope, imaging direction). Thus, in the imaging system and imaging method of the present embodiment, an XYZ orthogonal coordinate system is used in which the moving direction of the moving body (moving device) is the X-axis, the direction intersecting with the moving direction is the Z-axis, and the direction perpendicular to the X-axis and Z-axis is the Y-axis.

[0010] The imaging system and imaging method of the present embodiment divide a slope (object) into a plurality of regions in the height direction, and when each divided slope region is imaged by a plurality of cameras (imaging units), an exposure time required for imaging with an appropriate amount of exposure is ensured. That is, when imaging a slope (object), an exposure time required for imaging with an appropriate amount of exposure for the brightness of the imaged image to be appropriate is ensured. Furthermore, the entire slope is imaged by moving not only (in addition to) the height direction of the slope, but also in the width direction of the slope, continuously imaging the slope, and stitching together the captured images. That is, the slope is also divided into a plurality of regions in the width direction and imaged.

[0011] In this specification, "imaging" may be read as "photographing." In addition, in this specification, a case is described in which the "target" is a slope and the "moving body (moving device)" is a vehicle, but the "target" may be something other than a slope, and the "moving body (moving device)" may be something other than a vehicle.

[0012] In this specification, the imaging system is a component including a first imaging unit (e.g., a first camera and its imaging sensor) and a second imaging unit (e.g., a second camera and its imaging sensor). In this specification, the first imaging unit (e.g., the imaging sensor of the first camera) and the second imaging unit (e.g., the imaging sensor of the second camera) do not mean that the number of imaging units (cameras (imaging sensors)) is two, but that there are a plurality of imaging units (cameras (imaging sensors)). Among the plurality of imaging units (cameras (imaging sensors)), the imaging unit (camera (imaging sensor)) having a relatively long imaging distance of the object (slope) is defined as the "first imaging unit (e.g., the imaging sensor of the first camera)," and the imaging unit (camera (imaging sensor)) having a relatively short imaging distance of the object (slope) is defined as the "second imaging unit (e.g., the imaging sensor of the second camera)."

[0013] In this specification, the "imaging reference position" is defined as follows. The imaging reference position means a position on the camera side for identifying the distance from the camera to the subject (object) during imaging. For example, the imaging surface of the imaging element, which is the imaging sensor of the camera, corresponds to the imaging reference position. For example, in Figs. 8A-8C described later, the "imaging reference position" is used with the intention of indicating the distance from the imaging surface (imaging reference surface) of the imaging element of the first camera to the first slope imaging area, and the distance from the imaging surface (imaging reference surface) of the imaging element of the second camera to the second slope imaging area. In other words, the installation positions of the cameras (first and second cameras) are not the imaging reference positions. In this embodiment, the first camera and the second camera cannot be installed at the same physical position, so they are installed differently in at least one of the horizontal and vertical directions. In reality, even if the imaging reference position of the first camera and the imaging reference position of the second camera are different, it is possible to correct them later. However, for the sake of convenience (to simplify the explanation), this specification will be described assuming that the imaging reference position of the first camera and the imaging reference position of the second camera are common. That is, the "imaging reference position (common imaging reference position) of the first imaging unit and the second imaging unit" is defined, and the distance from this imaging reference position (common imaging reference position) to the first and second imaging areas (first and second slope imaging areas) is specified. Furthermore, the first camera and the second camera will be described assuming that they are arranged in a line along the moving direction of the moving body or in a line along the height direction of the moving body so that the imaging reference position is common.

[0014] In this specification, the term "object" is defined as including, for example, a slope that is an artificially processed incline among road wooden structures, and the slope (object) is imaged to obtain an image to be used for inspection. Also, the "slope imaging area" means an area of ​​the slope of the object (object to be inspected) that is imaged for inspection. Also, in this embodiment, the "slope imaging area" is divided and imaged while moving, so the range imaged in one image is defined as the "imaging area". In this way, the "slope imaging area" and the "imaging area" are clearly distinguished from each other when they are defined and used.

[0015] In this specification, the term "imaging width" refers to the imaging range in the moving direction when the imaging section (imaging sensor) of the camera captures the imaging area in one image. In addition, when the imaging sensor is a line sensor, the "imaging width" is defined as the imaging range in the moving direction captured by one pixel in one image. In this embodiment, the imaging sensor is a line sensor, but the present invention is also applicable to the imaging sensor being an area sensor.

[0016] In this specification, "the second imaging area is closer to the imaging reference position than the first imaging area", more specifically, "the second imaging area is closer to the first imaging area in the direction intersecting the movement direction from the imaging reference positions of the first imaging unit and the second imaging unit" means that, when the relationship between the distance from the imaging reference position to the first imaging area and the distance from the imaging reference position to the second imaging area is specified, the latter is smaller than the former. In other words, "close / far" from the imaging reference position is a relative expression. In addition, in this specification, when an expression such as "the imaging distance of the first imaging area is long / far" or "the imaging distance of the second imaging area is short / near" is simply used, it means that "the second imaging area is closer to the first imaging area in the direction intersecting the movement direction from the imaging reference positions of the first imaging unit and the second imaging unit" described here.

[0017] <General configuration of condition inspection system (imaging system)> FIG. 1 is a diagram showing an example of the overall configuration of a state inspection system 1 for implementing the imaging system and imaging method of the present embodiment. The state inspection system 1 is an example of an information processing system, and is a system for inspecting the state of road earthwork structures using various data acquired by a mobile system 60. The mobile system 60 is defined as a mobile body (mobile device) equipped with an imaging system 7 and a Lidar sensor, which will be described later (a different component from the imaging system 7). Road earthwork structures are a general term for structures that are mainly made of ground materials such as soil and rocks, etc., constructed to build roads, and structures associated therewith, and refer to cut and slope stabilization facilities, embankments, culverts, and similar. In addition, artificially processed slopes such as cut and slope stabilization facilities, embankments, and culverts, and slopes of artificial structures are called slopes. In other words, slopes of road earthwork structures are called slopes. Hereinafter, road earthwork structures may be called "objects", "slope", and "objects including slopes".

[0018] The condition inspection system 1 is composed of a mobile system 60, an evaluation system 4, a terminal device 1100 of the national or local government, and a terminal device 1200 of a commissioned business operator. The mobile system 60 is an example of an imaging system, and is composed of a data acquisition device 9 and a mobile body (mobile device) 6 such as a vehicle equipped with the data acquisition device 9. The vehicle may be a vehicle that runs on a road or a vehicle that runs on a railroad. The data acquisition device 9 has an imaging system 7, which is an example of a measuring device that measures a structure, as well as a distance sensor 8a and a GNSS (Global Navigation Satellite System) sensor 8b. GNSS is a general term for satellite positioning systems such as the Global Positioning System (GPS) or the Quasi-Zenith Satellite System (QZSS).

[0019] The imaging system 7 includes the above-mentioned first imaging unit and second imaging unit that image a plurality of regions (slope imaging regions) divided in the height direction (Y direction) of the slope (object). The first imaging unit and second imaging unit each have a line camera equipped with a line sensor (imaging sensor) in which photoelectric conversion elements are arranged in one or more rows. The imaging system 7 images a position of the slope (object) along a predetermined imaging range on an imaging surface along the traveling direction (X-axis direction) of the mobile body 6. The imaging system 7 is not limited to a line camera, and may be a camera equipped with an area sensor (imaging sensor) in which photoelectric conversion elements are arranged in a planar shape.

[0020] The distance sensor 8a is a ToF (Time of Flight) sensor, and measures the distance to the subject imaged by the imaging system 7. The GNSS sensor 8b is a positioning means that receives signals transmitted at each time from a plurality of GNSS satellites, and calculates the distance to the satellite from the difference in the time at which each signal was received, thereby measuring a position on the earth. The positioning means may be a device dedicated to positioning, or may be an application dedicated to positioning installed in a PC (Personal Computer), a smartphone, or the like. The distance sensor 8a and the GNSS sensor 8b are examples of sensor devices. The distance sensor 8a is also an example of a three-dimensional sensor.

[0021] The ToF sensor used as the distance sensor 8a measures the distance from a light source to an object by irradiating the object with laser light from a light source and measuring the scattered or reflected light.

[0022] In this embodiment, the distance sensor 8a is a LiDAR (Light Detection and Ranging) sensor. LiDAR is a method of measuring the time of flight of light using pulses, but as another method of the ToF sensor, the distance may be measured by a phase difference detection method. In the phase difference detection method, a laser light amplitude-modulated at a fundamental frequency is irradiated to a measurement range, the reflected light is received, and the phase difference between the irradiated light and the reflected light is measured to obtain time, and the distance is calculated by multiplying the time by the speed of light. In addition, the distance sensor 8a may be configured by a stereo camera or the like.

[0023] By using a three-dimensional sensor, the mobile system 60 can obtain three-dimensional information that is difficult to obtain from a two-dimensional image, such as the height, inclination angle, or projection of a slope.

[0024] The mobile body system 60 may further include an angle sensor 8c. The angle sensor 8c is a gyro sensor or the like for detecting the angle (attitude) or angular velocity (or each acceleration) of the imaging direction of the imaging system 7.

[0025] The evaluation system 4 is constructed by an evaluation device 3 and a data management device 5. The evaluation device 3 and the data management device 5 constituting the evaluation system 4 can communicate with a mobile system 60, a terminal device 1100, and a terminal device 1200 via a communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), and the like. The communication network 100 may include not only wired communication but also networks using wireless communication such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution). The evaluation device 3 and the data management device 5 may also have a communication function using a short-range communication technology such as NFC (Near Field Communication) (registered trademark).

[0026] The data management device 5 is an example of an information processing device, and is a computer such as a PC that manages various data acquired by the data acquisition device 9. The data management device 5 receives various acquired data from the data acquisition device 9, and transfers the received various acquired data to the evaluation device 3 that performs data analysis. Note that the method of transferring the various acquired data from the data management device 5 to the evaluation device 3 may be manual transfer using a USB (Universal Serial Bus) memory or the like.

[0027] The evaluation device 3 is a computer such as a PC that evaluates the condition of the slope based on various acquired data transferred from the data management device 5. A dedicated application program for evaluating the condition of the slope is installed in the evaluation device 3. The evaluation device 3 detects the type or structure of the slope from the captured image data and sensor data, extracts shape data, and performs a detailed analysis by detecting the presence or absence of deformation and the degree of deformation. The evaluation device 3 also generates a report to be submitted to a road administrator such as a national government, a local government, or a commissioned business operator, using the captured image data and sensor data, the evaluation target data, and the detailed analysis results. The data of the report generated by the evaluation device 3 is submitted to the national government or a local government via the commissioned business operator in the form of electronic data or printed on a document. The report generated by the evaluation device 3 is called an investigation record sheet, an inspection sheet, an investigation ledger, or a report. The evaluation device 3 is not limited to a PC, and may be a smartphone or a tablet terminal. The evaluation system 4 may be configured to construct the evaluation device 3 and the data management device 5 as a single device or terminal.

[0028] The terminal device 1200 is provided in a commissioned business operator, and the terminal device 1100 is provided in a national or local government. The evaluation device 3, the terminal device 1100, and the terminal device 1200 are examples of communication terminals capable of communicating with the data management device 5, and various data managed by the data management device 5 can be viewed.

[0029] 2 is a diagram showing an example of a state of a slope being inspected using a state inspection system 1 for implementing the imaging system and imaging method of this embodiment. A mobile body system 60 causes a mobile body 6 equipped with a data acquisition device 9 to travel on a road while capturing images of a predetermined range of a slope (slope of a target object) with an imaging system 7.

[0030] As shown in Figure 2, a cut slope is called a cut slope, and a piled-up slope is called a bank slope. In addition, the slope on the side of a road that runs along the side of a mountain is called a natural slope. Cut slopes and bank slopes can be made more durable by planting plants on the surface, and can be left unchanged for decades. However, this is not always the case. When cut slopes, bank slopes, and natural slopes deteriorate due to wind and rain, surface collapses occur, causing rocks and soil to fall, or the mountain collapses, causing road closures. To prevent such situations, methods are used to spray mortar on the surface of the slope (mortar spraying) or to install and harden concrete structures to slow down the rate at which the slope deteriorates when exposed to wind and rain. Structures constructed using such methods are called earthwork structures. Earthwork structures include retaining walls that are installed between natural slopes and roads, and rockfall protection fences that prevent rocks from falling onto the road. Both of these are intended to prevent road closures or human injury due to the outflow of soil, sand, falling rocks, etc. onto the road.

[0031] In recent years, the deterioration of earthwork structures that have been in place for decades has become significant, and the development of social infrastructure has become a major issue. Therefore, it is important to detect the deterioration of earthwork structures early and to inspect and maintain them to ensure their longevity. Conventional inspections of natural slopes and earthwork structures involve visual inspections by experts to investigate rock falls, collapses, landslides, or debris flows on the slopes and to create repair plans.

[0032] However, visual inspections by experts have problems with efficiency, such as the inability to inspect all of the large number of earthwork structures throughout Japan in a given period of time and the inability to inspect embankments at high altitudes or along rivers. In addition, visual inspections are unable to quantitatively grasp the degree of deterioration, such as cracks or peeling, that has occurred on the surface of earthwork structures.

[0033] Therefore, the condition inspection system 1 according to the embodiment acquires captured image data of the slope of an earthwork structure by the imaging system 7, and acquires sensor data including three-dimensional information by three-dimensional sensors such as the distance sensor 8a. The evaluation system 4 then combines the acquired captured image data and sensor data to evaluate the condition of the slope, thereby detecting shape data indicating the three-dimensional shape of the slope and detecting abnormalities such as cracks and peeling. This enables the condition inspection system 1 to efficiently perform evaluations that are difficult to inspect visually by humans.

[0034] Fig. 3 shows an example of the state of a slope. Fig. 3(a) is an image showing the surface of the slope five years before the collapse, and Fig. 3(b) is an explanatory diagram of the image shown in Fig. 3(a). In this state, cracks in the surface layer of the slope are noticeable, and image analysis shown in a development diagram or the like is effective in detecting changes or signs of changes in the surface layer, such as cracks, peeling, and seepage.

[0035] Figure 3(c) is an image showing the surface of the slope two years before the collapse, and Figure 3(d) is an explanatory diagram of the image shown in Figure 3(c). In this state, the inside of the slope has turned to soil, the soil has pushed against the surface of the slope, and the slope has bulged. In order to detect three-dimensional deformations such as cracks, steps, and bulges, three-dimensional analysis of images such as development drawings + cross-sections is effective.

[0036] Fig. 3(e) is an image showing the surface of the collapsed slope, and Fig. 3(f) is an explanatory diagram of the image shown in Fig. 3(e). In this state, the surface layer of the slope is unable to contain the soil and has collapsed.

[0037] 4 is a diagram showing an example of a hardware configuration of the data acquisition device 9. The data acquisition device 9 includes a controller 900 that controls the processing or operation of the data acquisition device 9, together with the imaging system 7 and the sensor device 8 as shown in FIG.

[0038] The controller 900 includes an imaging device I / F (Interface) 901, a sensor device I / F 902, a bus line 910, a CPU (Central Processing Unit) 911, a ROM (Read Only Memory) 912, a RAM (Random Access Memory) 913, a HD (Hard Disk) 914, a HDD (Hard Disk Drive) controller 915, a network I / F 916, a DVD-RW (Digital Versatile Disk Rewritable) drive 918, a media I / F 922, an external device connection I / F 923 and a timer 924.

[0039] Of these, the imaging device I / F 901 is an interface for transmitting and receiving various data or information to and from the imaging system 7. The sensor device I / F 902 is an interface for transmitting and receiving various data or information to and from the sensor device 8. The bus line 910 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 911 shown in FIG.

[0040] Furthermore, the CPU 911 controls the operation of the entire data acquisition device 9. The ROM 912 stores programs used to drive the CPU 911, such as the IPL. The RAM 913 is used as a work area for the CPU 911. The HD 914 stores various data such as programs. The HDD controller 915 controls the reading and writing of various data from and to the HD 914 under the control of the CPU 911. The network I / F 916 is an interface for data communication using the communication network 100.

[0041] A DVD-RW drive 918 controls reading and writing of various data from and to a DVD-RW 917, which is an example of a removable recording medium. Note that the recording medium is not limited to a DVD-RW, and may be a DVD-R, a Blu-ray (registered trademark) Disc, or the like.

[0042] The media I / F 922 controls reading and writing (storing) data from and to a recording medium 921 such as a flash memory. The external device connection I / F 923 is an interface for connecting an external device such as an external PC 930 having a display, a reception unit, and a display control unit. The timer 924 is a measuring device having a time measurement function. The timer 924 may be a software timer by a computer. The timer 924 is preferably synchronized with the time of the GNSS sensor 8b. This makes it easy to synchronize the time and associate the positions of each sensor data and captured image data.

[0043] FIG. 5 is a diagram for explaining a captured image acquired by the mobile body system.

[0044] The mobile body system 60 captures an image of a slope on a road using the imaging system 7 provided in the data acquisition device 9 while the mobile body 6 is traveling. The X-axis direction shown in Fig. 5 indicates the direction of movement of the mobile body 6, the Y-axis direction is the vertical direction, and the Z-axis direction is perpendicular to the X-axis and Y-axis directions and indicates the depth direction from the mobile body 6 toward the slope.

[0045] As the mobile unit 6 travels, the data acquisition device 9 acquires the captured image 1, the distance measurement image 1, the captured image 2, and the distance measurement image 2 in chronological order as shown in Fig. 5. The distance measurement image 1 and the distance measurement image 2 are images acquired by the distance sensor 8a. At this time, the imaging system 7 and the sensor device 8 are time-synchronized, and the captured image 1 and the distance measurement image 1, and the captured image 2 and the distance measurement image 2 are images of the same area of ​​the slope. In addition, the tilt of the captured image is corrected (image correction) based on the attitude of the vehicle at the time of capture, and the image data and the positioning data (north latitude and east longitude) are linked based on the time of the captured image.

[0046] In this way, the mobile body system 60 acquires captured image data of the slope and sensor data acquired in response to imaging by the imaging system 7 while driving the vehicle as the mobile body 6, and uploads them to the data management device 5. Note that the data acquisition device 9 may acquire the ranging images and the captured images while driving separately, but considering changes in the slope shape due to collapse, etc., it is preferable to acquire the ranging images and the captured images for the same slope shape while driving at the same time.

[0047] FIG. 6 is an explanatory diagram of a captured image and a distance measurement image.

[0048] Fig. 6(a) shows captured image data 7A of captured images 1, 2, etc. shown in Fig. 5. Each pixel 7A1 of the captured image data 7A acquired by the imaging system 7 is arranged at coordinates corresponding to the X-axis direction and the Y-axis direction shown in Fig. 5, and has luminance information corresponding to the amount of stored power. In other words, the captured image data 7A is an example of a luminance image.

[0049] Then, the luminance information of each pixel 7A1 of the captured image data 7A is associated with coordinates corresponding to the X-axis direction and the Y-axis direction shown in FIG. 5, and is stored in the storage unit as captured image data.

[0050] Fig. 6(b) shows distance measurement image data 8A such as distance measurement images 1 and 2 shown in Fig. 5. Each pixel 8A1 of the distance measurement image data 8A acquired by the distance sensor 8a is arranged at coordinates corresponding to the X-axis direction and the Y-axis direction shown in Fig. 5, and has distance information in the Z-axis direction shown in Fig. 5 corresponding to the amount of stored power. Note that although the distance measurement image data 8A is three-dimensional point cloud data, it is generally referred to as distance measurement image data because it is visually displayed with luminance information added when it is visually recognized by a user. The captured image data 7A and distance measurement image data 8A are collectively referred to as image data.

[0051] Then, the distance information of each pixel 8A1 of the distance measurement image data 8A is stored in the storage unit as three-dimensional data included in the sensor data in association with coordinates corresponding to the X-axis and Y-axis directions shown in FIG.

[0052] Here, since the captured image data 7A shown in FIG. 6(a) and the distance measurement image data 8A shown in FIG. 6(b) are images of the same area of ​​the slope, the brightness information and distance information are stored in the memory unit in correspondence with the coordinates corresponding to the X-axis and Y-axis directions shown in the figure.

[0053] 7 is an explanatory diagram of a plurality of imaging regions. As shown in Fig. 7(a), the imaging system 7 moves together with the moving body 6 and captures images of a slope imaging region 70 of a slope 80 divided into a plurality of imaging regions d11, d12, ... at a constant imaging interval t along the X-axis direction which is the moving direction of the moving body 6.

[0054] As shown in FIG. 7(b), the captured image of multiple imaging regions d11, d12, etc. is a slit-shaped captured image that is long in the Y-axis direction, and by stitching together the images of the multiple imaging regions d11, d12, etc., an image of the slope imaging region 70 that is continuous in the X-axis direction can be obtained.

[0055] Fig. 7(c) is a diagram showing a case where the entire slope imaging area 70 is divided into a plurality of slope imaging target areas and imaged when the entire slope imaging area 70 is imaged. In Fig. 7(c), the entire slope imaging area 70 is imaged by dividing the image into four slope imaging target areas 701A, 702A, 701B, and 702B. That is, when imaging the slope imaging area 70, the slope imaging area 70 is divided into a plurality of areas and imaged, and each of the divided areas is called the slope imaging target area 701A, 702A, 701B, and 702B.

[0056] 7(b), each of the multiple slope imaging target areas 701A, 702A, 701B, and 702B is divided into multiple imaging areas d11, d12, etc., and the images of the multiple imaging areas d11, d12, etc. are stitched together to obtain images of the multiple slope imaging target areas 701A, 702A, 701B, and 702B. Then, the images of the multiple slope imaging target areas 701A, 702A, 701B, and 702B are stitched together to obtain an image of the entire slope imaging area 70.

[0057] In this case, the imaging system 7 includes a plurality of imaging units (the above-mentioned first imaging unit and second imaging unit), and the slope imaging target areas 702A and 702B are imaged by an imaging unit different from the imaging unit that images the slope imaging target areas 701A and 701B. For example, the slope imaging target areas 701A and 701B may be defined as a "second imaging area" arranged at a low position in the height direction (Y-axis direction), and the second imaging area may be imaged by a "second camera (second imaging unit)" that is responsible for a low position in the height direction (Y-axis direction). Also, the slope imaging target areas 702A and 702B may be defined as a "first imaging area" arranged at a high position in the height direction (Y-axis direction), and the first imaging area may be imaged by a "first camera (first imaging unit)" that is responsible for a high position in the height direction (Y-axis direction). Also, the "first imaging area" may be defined as an imaging area of ​​the slope (object) that is far from the imaging reference position (common imaging reference position) which is the imaging surface of the first camera (first imaging unit), and the "second imaging area" may be defined as an imaging area of ​​the slope (object) that is close to the imaging reference position (common imaging reference position) which is the imaging surface of the second camera (second imaging unit). Also, as described above, in this specification, the imaging reference position which is the imaging surface of the first camera (first imaging unit) and the second camera (second imaging unit) will be described as being the common imaging reference position.

[0058] In addition, the slope imaging target area 701B may be imaged by the same imaging unit that images the slope imaging target area 701A, and the slope imaging target area 702B may also be imaged by the same imaging unit that images the slope imaging target area 702A.

[0059] As shown in Figure 7(a), at the same time that the imaging system 7 captures the slope imaging area of ​​the slope 80 by dividing it into multiple imaging areas d11, d12, etc., it is desirable that the distance sensor 8a also acquires distance information indicating the distance from the distance sensor 8a to each of the multiple imaging areas d11, d12, etc.

[0060] 6, it is possible to easily associate the luminance information of each pixel 7A1 of the captured image data 7A acquired by the imaging system 7 with the distance information of each pixel 8A1 of the distance measurement image data 8A acquired by the distance sensor 8a. By associating the luminance information of each image of the captured image capturing the slope imaging region of the slope 80 with the distance information of each pixel of the distance measurement image capturing the slope imaging region of the slope 80, it is possible to perform a highly accurate inspection of the slope imaging region of the slope 80.

[0061] In this way, the state inspection system (imaging system) 1 of this embodiment images the slope while a mobile body (vehicle) equipped with an imaging unit (camera) travels along a road beside the slope. At this time, a plurality of imaging units (cameras) are prepared, and the slope is divided in the moving direction (X-axis direction) of the mobile body (vehicle) and imaged by each imaging unit (camera). Also, the imaging timing of each imaging unit (camera) is synchronized to divide and image the area in the height direction (Y-axis direction) of the slope. In this way, the slope is sliced ​​in the moving direction (X-axis direction) of the mobile body (vehicle) and imaged, and the sliced ​​images are joined together after imaging, so that a surface image of the entire slope can be obtained with a predetermined resolution. This makes it possible to inspect the slope for deformation, and by reducing the joined images, it becomes possible to check the condition of a wide range of the slope.

[0062] <Technical issues with conventional condition inspection systems (imaging systems)> Here, we consider the case where multiple slope imaging areas are defined by dividing the slope (object) in the height direction (Y-axis direction), and the first slope imaging area farther from the moving body is imaged by the first camera, and the second slope imaging area closer to the moving body is imaged by the second camera (where the imaging distances of the slope by the first camera and the second camera are different, and the imaging distance of the slope by the first camera is relatively long and the imaging distance of the slope by the second camera is relatively short).

[0063] 8A, 8B, and 8C are diagrams that show the difference between the imaging distance (first imaging distance) of the first slope imaging area by the first camera and the imaging distance (second imaging distance) of the second slope imaging area by the second camera. FIG. 8A shows a typical shape often seen on an actual slope, and can be said to be the result of adding the imaging distances of the first camera and the second camera to the already-mentioned FIG. 2. FIG. 8B shows an explanation assuming that the slope is vertical, since it is difficult to explain the imaging distances of the first camera and the second camera when the slope is an inclined surface as in FIG. 8A. In addition, in FIG. 8A and FIG. 8B, the step portions of the first slope imaging area and the second slope imaging area are horizontal, which is a simplification, and it is not important whether the step portions are horizontal or not. FIG. 8C is a drawing showing FIG. 8B from above (XZ plane) with the step portion omitted, and by showing it from above (XZ plane), it illustrates a "first imaging distance" based on the "imaging reference plane (imaging reference position)" and the "first slope imaging area", and a "second imaging distance" based on the "imaging reference plane (imaging reference position)" and the "second slope imaging area". In FIG. 8A to FIG. 8C, the imaging distance (first imaging distance) of the first slope imaging area by the first camera is larger (longer) than the imaging distance (second imaging distance) of the second slope imaging area by the second camera. In FIG. 8A to FIG. 8C, the arrows indicating the imaging distance of the first camera and the imaging distance of the second camera are common to the lens optical axis of the first camera and the lens optical axis of the second camera, respectively (they can be read as interchangeable).

[0064] As described above, there are many different shapes of slopes as objects, and the shapes are not limited to those shown in Figs. 8A-8C. However, in this embodiment, it is an essential prerequisite that the imaging distance of the first slope imaging area by the first camera is greater (longer) than the imaging distance of the second slope imaging area by the second camera. Also, the imaging distance is not necessarily long at the high points of the slope, and short at the low points of the slope. For example, there may be cases where the imaging distance is short at the high points of the slope, and long at the low points of the slope (such a slope point).

[0065] Under the above preconditions, it is assumed that the first and second cameras are of the same specifications (same standards), i.e., cameras with the same focal length of the imaging lens, size of the imaging sensor, and angle of view. In this case, continuous images are taken while moving in the width direction (X-axis direction) of the slope, which is the moving direction of the moving object, and the captured images are stitched together, so that in one image capture, the imaging width (width in the X-axis direction) of the second camera, which has a shorter imaging distance, is smaller (shorter) than the imaging width (width in the X-axis direction) of the first camera, which has a longer imaging distance.

[0066] Due to the relationship between the imaging widths (widths in the X-axis direction) of the first and second cameras, the imaging interval of the second camera is smaller (shorter) than the imaging interval of the first camera. Also, as described above, since the slope is divided in the traveling direction (X-axis direction) and images are captured and then joined together after imaging, the imaging interval of the first camera cannot be shorter than the exposure time of the first camera. Similarly, the imaging interval of the second camera cannot be shorter than the exposure time of the second camera.

[0067] Here, the first camera and the second camera are mounted on a moving body (vehicle) and move together in the same direction (X-axis direction), i.e., they have the same moving speed. Since the first camera and the second camera are independent cameras, it is theoretically possible to make the image capture intervals different. However, since the first camera and the second camera are mounted on a common moving body (vehicle), the exposure time of the first camera and the exposure time of the second camera affect the moving speed of the moving body. In other words, the longer of the exposure time of the first camera and the exposure time of the second camera becomes the bottom bottleneck for the moving speed of the moving body (vehicle).

[0068] Specifically, the imaging interval of the second camera becomes shorter than that of the first camera, and the upper limit of the exposure time of the second camera becomes shorter than that of the first camera. Therefore, for example, the lower limit of the brightness of the subject (slope) that can be imaged with proper exposure becomes higher in the second camera, and if the moving body moves at a desired speed, the exposure time for the second camera to image with an appropriate amount of exposure may not be secured. In other words, if the brightness of the subject (second slope area) imaged by the second camera is low, the exposure of the image captured by the second camera may be insufficient, resulting in a dark image. Therefore, it may not be possible to move the moving body at a desired speed. In addition, with regard to the first camera, it is necessary to avoid overlapping of the range imaged in one image in order to stitch the images captured. Therefore, it becomes necessary to widen the imaging interval when the moving speed of the moving body slows down.

[0069] In particular, when the second slope imaging area, which has a shorter imaging distance, is located below the first slope imaging area, which has a longer imaging distance, the second slope imaging area is likely to be less exposed to sunlight than the first slope imaging area (e.g., it is in the shade), and the second slope imaging area is likely to have lower brightness than the first slope imaging area. In this case, the second camera is more likely to be underexposed.

[0070] FIG. 9A is a diagram showing the relationship between the imaging width and the imaging interval when the first camera and the second camera have the same specifications. FIG. 9B is a diagram showing the imaging width (resolution of the captured image) per pixel in an image of a slope. FIG. 9C is a diagram showing the imaging interval when continuously capturing images, and is a diagram explaining that the imaging width and the imaging interval are related (proportional). As shown in FIG. 9A and FIG. 9B, the imaging width of the second camera capturing the second slope imaging area with a short imaging distance is smaller (shorter) than the imaging width of the first camera capturing the first slope imaging area with a long imaging distance. Also, as shown in FIG. 9C, the upper limit of the imaging interval of the second camera capturing the second slope imaging area with a short imaging distance is smaller (shorter) than the upper limit of the imaging interval of the first camera capturing the first slope imaging area with a long imaging distance. In addition, in FIG. 9A, the first camera and the second camera are shown aligned in the movement direction (X-axis direction) to show that the imaging width differs depending on the imaging distance, but the first camera and the second camera do not need to be aligned in the movement direction (X-axis direction) and may be aligned in the height direction (Y-axis direction).

[0071] <Technical Concept of the Invention> The inventor of the present invention regarded the above-mentioned problem as an important technical issue, and reconsidered the conventional technical common sense of using cameras of the same specifications (same standards) for the first camera that captures the first slope imaging area with a long imaging distance and the second camera that captures the second slope imaging area with a short imaging distance, and came up with the idea of ​​capturing images of the first slope imaging area and the second slope imaging area with different imaging distances with appropriate exposure, thereby completing the present invention. Here, the technical concept (overview) of the present invention will be described.

[0072] In this embodiment, the slope is divided in the height direction and photographed, a first imaging area that is far from the imaging reference position is photographed with a first camera, and a second imaging area that is close to the imaging reference position is photographed with a second camera. The first imaging area and the second imaging area are also divided in the slope width direction and photographed by the first camera and the second camera, respectively.

[0073] Regarding the imaging width for imaging the slope width direction in each of the first and second cameras, if the parameters that determine the imaging width, the focal length of the imaging lens and the size in the imaging direction of the imaging sensor (more specifically, the size of one pixel in the imaging direction of the imaging sensor), are the same, the imaging width of the second camera imaging the second imaging area located closer to the imaging reference position will be narrower than the imaging width of the first camera imaging the first imaging area located farther from the imaging reference position, and it will not be possible to secure the exposure time required to capture an image with an appropriate exposure amount to ensure appropriate brightness.

[0074] To solve this problem, the size of the image sensor in the imaging direction, which is a parameter that determines the imaging width, is increased so that the imaging width of the second camera is within a predetermined range of the imaging width of the first camera (preferably so that the imaging width of the second camera approaches the imaging width of the first camera) by making the image size of the second camera larger than the image size of the first camera. By increasing the imaging width, the imaging interval (time) when continuously photographing in the moving direction is extended, and as a result, it is possible to ensure the exposure time required to photograph with an appropriate exposure amount so that the luminance of the photographed image is appropriate.

[0075] In this way, one of the key features of the present invention is the means of increasing "the size in the imaging direction of the imaging sensor of the second camera (more specifically, the size of one pixel in the imaging direction of the imaging sensor)", the effect of extending the shooting interval (time) when taking continuous images in the moving direction by widening the imaging width of the second camera so that it is within a predetermined range of the imaging width of the first camera, and the effect of ensuring the exposure time necessary to capture images with an appropriate exposure amount so that the captured images have appropriate brightness.

[0076] Here, the fact that the shooting interval (time) during continuous shooting is extended when the imaging width is widened is premised on the assumption that the imaging ranges do not overlap during continuous shooting and that there is no omission of the imaging range in the moving direction. Also, it can be said unequivocally that the shooting interval (time) is extended when the imaging width (distance) is widened, but the degree to which the shooting interval (time) is extended in response to the widening of the imaging width (distance) is related to the moving speed of the moving object.

[0077] Regarding the relationship between the shooting interval (time) and the exposure time, under the condition that the shooting ranges do not overlap and there is no omission or omission of the shooting range in the movement direction, the exposure time cannot be longer than the shooting interval (time). In other words, the upper limit of the exposure time is shorter than the shooting interval (time).

[0078] Under the above prerequisites and conditions, in the embodiment, the imaging width (distance) is widened to extend the imaging interval (time), and as a result, the upper limit of the exposure time is extended, so that the necessary exposure amount can be secured. Note that the physical quantity of the imaging width is distance, the physical quantity of the imaging interval is time, and the physical quantity of the exposure time is time. For this reason, the imaging interval and the exposure time can be directly compared because they have the same physical quantity, but in order to convert the imaging width, whose physical quantity is distance, into the imaging interval, whose physical quantity is time, a moving speed, which is a physical quantity called speed, is required.

[0079] In the slope inspection of this embodiment, the resolution of the camera (e.g., 16K camera) is selected and set to capture the imaging area farthest from the imaging reference position at a desired resolution. When capturing images with a lens of the same resolution and focal length, the resolution of the captured image becomes finer as the imaging area approaches the imaging reference position, so in order to capture the imaging area at a desired resolution, the imaging element of the camera and the focal length of the photographing lens are selected and set based on the imaging area farthest from the imaging reference position.

[0080] In this specification, the imaging area farther from the imaging reference position is described as the first imaging area, and the imaging area closer to the imaging reference position is described as the second imaging area. The parameters of the first camera that captures the first imaging area are used as a reference, and the parameters of the second camera that captures the second imaging area, which is an imaging area closer to the imaging reference position than the first imaging area, are set.

[0081] Here, "close / far" from the imaging reference position is a relative expression. In the case of specifying the magnitude relationship between the distance from the imaging reference position to the first imaging area and the distance from the imaging reference position to the second imaging area, for example, when the latter is smaller than the former, it is expressed as "the second imaging area is closer to the imaging reference position than the first imaging area", or more specifically, "the second imaging area is closer to the first imaging area in the direction intersecting the movement direction from the imaging reference positions of the first imaging unit and the second imaging unit". In addition, in this specification, when it is simply said that "the imaging distance of the first imaging area is long / far" or "the imaging distance of the second imaging area is short / near", it means that "the second imaging area is closer to the first imaging area in the direction intersecting the movement direction from the imaging reference positions of the first imaging unit and the second imaging unit".

[0082] The imaging system of this embodiment is an imaging system having a first imaging unit (first camera) and a second imaging unit (second camera). The imaging system is installed on a moving device and continuously images an object while moving in a moving direction. The first imaging unit images a first imaging area which is a part of the object (slope), and the second imaging unit images a second imaging area which is a part of the object (slope). The second imaging area is closer to the imaging reference position of the first imaging unit and the second imaging unit in a direction intersecting with the moving direction than the first imaging area. Here, the imaging width which is the imaging range in the moving direction when the first imaging unit images the first imaging area in one imaging is defined as the first imaging width, and the imaging width which is the imaging range in the moving direction when the second imaging unit images the second imaging area in one imaging is defined as the second imaging width. Then, a first imaging parameter for determining the first imaging width in the first imaging unit and a second imaging parameter for determining the second imaging width in the second imaging unit are made different from each other so that the second imaging width is within a predetermined range of the first imaging width.

[0083] More specifically, the first imaging parameter is the size of one pixel in the movement direction of a first imaging sensor possessed by the first imaging section, and the second imaging parameter is the size of one pixel in the movement direction of a second imaging sensor possessed by the second imaging section, and by making the size of one pixel in the movement direction of the second imaging sensor larger than the size of one pixel in the movement direction of the first imaging sensor, the second imaging width can be set within a predetermined range of the first imaging width.

[0084] Within the above-mentioned predetermined range, the second imaging width is preferably within a range of 50% to 100% of the first imaging width, and more preferably within a range of 70% to 100% of the first imaging width.

[0085] The first imaging parameter for determining the first imaging width in the first imaging unit and the second imaging parameter for determining the second imaging width in the second imaging unit are made different so that the second imaging width is within a predetermined range of the first imaging width, more specifically, the size of one pixel in the moving direction of the second imaging sensor is made larger than the size of one pixel in the moving direction of the first imaging sensor, thereby ensuring the exposure time required for capturing images with an appropriate exposure amount when capturing images with the first camera and the second camera. Also, distortion (e.g., curvature of field, etc.) is prevented from occurring in the captured image of the second camera, and a high-quality captured image can be obtained as a whole imaging system.

[0086] The above-mentioned effects are more significantly achieved by satisfying the condition that the second imaging width is within a range of 50% to 100% of the first imaging width, more preferably within a range of 70% to 100% of the first imaging width, as the above-mentioned predetermined range. In other words, by making the second imaging width as close as possible to the first imaging width (preferably making them the same), it is possible to achieve the maximum effects.

[0087] In order to widen the imaging width of the second camera, it is possible to shorten the focal length of the imaging lens of the second camera, to increase the size of the imaging sensor of the second camera, or to shorten the focal length of the imaging lens of the second camera and increase the size of the imaging sensor of the second camera. However, shortening the focal length of the imaging lens of the second camera tends to cause distortion of the imaging lens and, in turn, distortion (e.g., field curvature, etc.) in the captured image of the second camera, so it is preferable to adopt a method of increasing the size of the imaging sensor of the second camera. However, it is not intended to intentionally exclude the method of shortening the focal length of the imaging lens of the second camera.

[0088] The size of the imaging sensor of the first camera is a size necessary for imaging the first slope imaging area, which is the imaging target, at a desired resolution (resolution of the captured image). When the first camera and the second camera have the same specifications, the second slope imaging area is imaged at a higher resolution than the first slope imaging area. When the imaging width of the second camera is expanded to the imaging width of the first camera by increasing the size of the imaging sensor of the second camera, the resolution at which the first slope imaging area is imaged and the resolution at which the second slope imaging area is imaged may become the same (or approximately the same).

[0089] The first and second cameras (first and second imaging units) can be arranged in a row along the movement direction (e.g., the X-axis direction), or can be arranged in a row along a height direction (e.g., the Y-axis direction) that intersects both the movement direction (e.g., the X-axis direction) and a direction that intersects (e.g., the Z-axis direction).

[0090] The first and second imaging regions (e.g., the first slope imaging region and the second slope imaging region) can be disposed at different positions in the moving direction (e.g., the X-axis direction) and the height direction (e.g., the Y-axis direction) of the object. Specifically, the first imaging region (e.g., the first slope imaging region) can be disposed at a relatively high position in the height direction (e.g., the Y-axis direction), and the second imaging region (e.g., the second slope imaging region) can be disposed at a relatively low position in the height direction (e.g., the Y-axis direction).

[0091] 10A and 10B are diagrams showing the relationship between the focal length, the size of the image sensor (pixel size in the imaging width direction), and the imaging width, which are the premise of the present invention. In Fig. 10A and Fig. 10B, the focal length of the lens is the same, but the pixel size (size in the X-axis direction) of the image sensor is larger in Fig. 10B than in Fig. 10A, and the imaging width is larger (wider) in Fig. 10B than in Fig. 10A. The reason why the imaging width is larger (wider) in Fig. 10B than in Fig. 10A is that the size of the image sensor (pixel size in the imaging width direction) is larger in Fig. 10B than in Fig. 10A.

[0092] 11A, 11B, and 11C are conceptual diagrams of imaging in the first and second cameras of this embodiment. As shown in FIG. 11A and FIG. 11B, the imaging width of the second camera imaging the second slope imaging region with a short imaging distance is the same (almost the same) as the imaging width of the first camera imaging the first slope imaging region with a long imaging distance. Also, as shown in FIG. 11C, the upper limit of the imaging interval of the second camera imaging the second slope imaging region with a short imaging distance is the same (almost the same) as the upper limit of the imaging interval of the first camera imaging the first slope imaging region with a long imaging distance. Also, in FIG. 11A, the first and second cameras are shown aligned in the moving direction (X-axis direction), but the first and second cameras do not need to be aligned in the moving direction (X-axis direction) and may be aligned in the height direction (Y-axis direction). In this way, the pixel size in the imaging width direction of the imaging sensor of the second camera, which images a nearby area, is made larger so that the imaging widths of the first camera and the second camera are the same (approximately the same). In addition, by making the imaging widths the same (approximately the same), it becomes possible to make the imaging intervals the same (approximately the same).

[0093] <Numerical Example 1> 12A and 12B are diagrams illustrating an imaging system according to Numerical Example 1. FIG.

[0094] As shown in FIG. 12A and FIG. 12B, two cameras, that is, a first camera 71 as a first imaging unit and a second camera 72 as a second imaging unit, are mounted on the ceiling of a moving body (vehicle). The first camera 71 and the second camera 72 are arranged in a row along the X-axis direction, but may be arranged in a row along the Y-axis direction. The first camera 71 and the second camera 72 may be arranged with their positions shifted in the Z-axis direction. Here, as an example, a description will be given assuming that the traveling speed of the moving body (vehicle) in the X-axis direction is 30 mk / h. Also, the imaging reference position (common imaging reference position), which is the imaging plane of the first camera 71 and the second camera 72, is common in that it is the same position in the Z-axis direction.

[0095] In Numerical Example 1, the following numerical values ​​(parameters) are described for each of the first camera 71 and the second camera 72 (there are some overlapping and common points). Fig. 13 is a diagram for visually understanding the imaging parameters of the imaging unit (camera), and will be described with reference to Fig. 13 as appropriate. Imaging distance D from the imaging reference position to the imaging area (slope imaging area) (Distance D1 from the imaging reference position to the first slope imaging area, distance D2 from the imaging reference position to the second slope imaging area) Imaging width Y when imaging slope (Imaging width Y1 of the first slope imaging area, imaging width Y2 of the second slope imaging area) Resolution (number of pixels) when capturing images of slopes (Resolution (number of pixels) of the first sensor of the first camera, resolution (number of pixels) of the second sensor of the second camera) - Pixel size Y' of the camera's line sensor (image sensor) (Pixel size Y1' of the first sensor of the first camera, pixel size Y2' of the second sensor of the second camera) · The focal length f of the camera's imaging lens and the imaging angle θ (focal length f1 of the first imaging lens of the first camera, imaging angle of view θ1, focal length f2 of the second imaging lens of the second camera, imaging angle of view θ2) - Camera capture interval (1st camera imaging interval, 2nd camera imaging interval)

[0096] Fig. 14 is a diagram showing imaging parameters of the first camera and the second camera in Numerical Example 1. With reference to Fig. 14, imaging parameters of the first camera and the second camera in Numerical Example 1 are shown below. Distance D1 from the imaging reference position to the first slope imaging area: 8.0 m Distance D2 from the imaging reference position to the second slope imaging area: 2.5 m Imaging width Y1 of the first slope imaging area: 1.1 mm Imaging width Y2 of the second slope imaging area: 1.1 mm Pixel size Y1' of the first sensor of the first camera: 5.0 μm Pixel size Y2' of the second sensor of the second camera: 15.0 μm The focal length of the first imaging lens of the first camera is f1: 35.0 mm Imaging angle of the first imaging lens of the first camera θ1: 97.6° The focal length of the second imaging lens of the second camera is f2: 35.0 mm Imaging angle of the second imaging lens of the second camera θ2: 46.4° First camera imaging interval: 0.00014 seconds per pixel Second camera imaging interval: 0.00013 seconds per pixel

[0097] The following relational expression (approximation) holds between the above Y (Y1, Y2), D (D1, D2), f (f1, f2), and Y' (Y1', Y2'). Y = (Df) / f × Y' Y1 = (D1 - f1) / f1 x Y1' Y2 = (D2 - f2) / f2 × Y2'

[0098] In general, when the imaging width Y during imaging of a slope and the pixel size Y' of the camera's line sensor (imaging sensor) are specified, as well as the lateral magnification β of the camera's imaging lens, Y=Y'×β holds.

[0099] In addition to the above, when the imaging distance (from the front principal point to the slope) S of the camera's imaging lens is specified in addition to the focal length f of the camera's imaging lens, Y=(Sf) / f×Y' holds.

[0100] Here, on the slope side, the distance between a point on the lens surface and the object-side principal point on the optical axis is sufficiently smaller than the distance from the object-side principal point to the slope, so S can be approximated to D, and Y = (Sf) / f × Y' can be approximated to (equal to) Y = (Df) / f × Y'.

[0101] Fig. 15A and Fig. 15B are schematic diagrams illustrating the structure of a line sensor (image sensor), Fig. 15A is a diagram of the line sensor standing in the direction of gravity and seen from above, and Fig. 15B is a diagram of the line sensor seen from the front. The resolutions when imaging the first slope imaging area and the second slope imaging area will be described with reference to Fig. 15A and Fig. 15B.

[0102] When the resolution is defined as the number of pixels, the number of pixels L1 in the line direction and the number of pixels H1 in the running direction of the line sensor (image sensor) of the first camera, and the number of pixels L2 in the line direction and the number of pixels H2 in the running direction of the line sensor (image sensor) of the second camera are as follows: L1: 16000 pixels H1:1 pixel L2: 2000 pixels H2:1 pixel

[0103] If pixel pitch is added to the definition of resolution, the pixel pitch in the line direction of the line sensor (image sensor) in the first camera is 5μm, and the pixel pitch in the running direction is 5μm (on the slope side, the pixel pitch in the line direction is 1.1mm, and the pixel pitch in the running direction is 1.1mm). Also, the pixel pitch in the line direction of the line sensor (image sensor) in the second camera is 15μm, and the pixel pitch in the running direction is 15μm (on the slope side, the pixel pitch in the line direction is 1.1mm, and the pixel pitch in the running direction is 1.1mm).

[0104] <Numerical Example 2> In Numerical Example 1, the second camera is a 2K camera (2000 pixels) with a pixel size of 15 μm, so that the imaging widths of the first and second cameras are almost the same, and the exposure of the second camera is maximized. Numerical Example 2 differs from Numerical Example 1 in that the second camera is a 4K camera (4000 pixels) with a pixel size of 11 μm.

[0105] The imaging parameters of the first and second cameras in the second numerical example are as follows: Distance D1 from the imaging reference position to the first slope imaging area: 8.0 m Distance D2 from the imaging reference position to the second slope imaging area: 2.5 m Imaging width Y1 of the first slope imaging area: 1.1 mm Imaging width Y2 of the second slope imaging area: 0.8 mm Pixel size Y1' of the first sensor of the first camera: 5.0 μm Pixel size Y2' of the second sensor of the second camera: 11.0 μm The focal length of the first imaging lens of the first camera is f1: 35.0 mm Imaging angle of the first imaging lens of the first camera θ1: 97.6° The focal length of the second imaging lens of the second camera is f2: 35.0 mm Imaging angle of the second imaging lens of the second camera θ2: 64.3° First camera imaging interval: 0.00014 seconds per pixel Second camera imaging interval: 0.00009 seconds per pixel

[0106] When the resolution is defined as the number of pixels, the number of pixels L1 in the line direction and the number of pixels H1 in the running direction of the line sensor (image sensor) of the first camera, and the number of pixels L2 in the line direction and the number of pixels H2 in the running direction of the line sensor (image sensor) of the second camera are as follows: L1: 16000 pixels H1:1 pixel L2: 4000 pixels H2:1 pixel

[0107] <Numerical Example 3> 16A and 16B are diagrams showing an imaging system of Numerical Example 3. FIG. 17 is a diagram showing imaging parameters of the first camera and the second camera (the second X camera and the second Y camera) in Numerical Example 3. In Numerical Example 3, the second camera 72 includes the second X camera 72X and the second Y camera 72Y. That is, the system includes three cameras including the first camera 71, the second X camera 72X, and the second Y camera 72Y. The first camera 71, the second X camera 72X, and the second Y camera 72Y are arranged in this order in the X-axis direction, which is the traveling direction of the moving body (vehicle), and are positioned in the same (common) position in the Z-axis direction.

[0108] In Numerical Example 1, the first camera 71 captures an image at a distance of 8.0 m (8.0 m from the first lens), and the second camera 72 captures an image at a distance of 2.5 m (2.5 m from the second lens). The angle of view of the first lens of the first camera 71 is 97.6°, and the angle of view of the second lens of the second camera 72 is 46.4°. This camera angle of view ensures that no part of the slope is captured by the first camera 71 and the second camera 72 in the gravity direction (Y-axis direction). On the other hand, when capturing an image at a distance of 14 m by the first camera 71, the height of the slope is higher than the distance of 8.0 m, so that there is a range between the first camera 71 and the second camera 72 that cannot be captured.

[0109] Therefore, in the third numerical embodiment, a third camera is installed between the two cameras to fill in the missing area. In other words, the second camera 72 is divided into a second X camera 72X and a second Y camera 72Y. The imaging lens of the second X camera 72X has a focal length of 50 mm, making it easy to see small abnormalities.

[0110] The imaging parameters of the first camera 71, the second X camera 72X, and the second Y camera 72Y in the third numerical example are as follows. Distance D1 from the imaging reference position to the first slope imaging area: 14.0 m Distance from the imaging reference position to the 2nd X slope imaging area D2X: 6.5 m Distance from the imaging reference position to the 2nd Y slope imaging area D2Y: 2.5 m Imaging width Y1 of the first slope imaging area: 2.0 mm Imaging width of 2nd X-side slope imaging area Y2X: 1.9 mm Imaging width of 2nd Y slope imaging area Y2Y: 1.1 mm Pixel size Y1' of the first sensor of the first camera: 5.0 μm Pixel size Y2X' of the second sensor of the second X camera: 15.0 μm Pixel size Y2Y' of the second sensor of the second Y camera: 15.0 μm The focal length of the first imaging lens of the first camera is f1: 35.0 mm Imaging angle of the first imaging lens of the first camera θ1: 97.6° Focal length of the 2X imaging lens of the 2X camera f2X: 50.0 mm Imaging angle of view of the 2X imaging lens of the 2X camera θ2X: 33.4° Focal length f2Y of the 2nd Y imaging lens of the 2nd Y camera: 35.0 mm Imaging angle of view of the 2Y imaging lens of the 2Y camera θ2Y: 46.4° First camera imaging interval: 0.00024 seconds per pixel Imaging interval of 2X camera: 0.00023 seconds per pixel Imaging interval of the second Y camera: 0.00013 seconds per pixel

[0111] When the resolution is defined as the number of pixels, the number of pixels L1 in the line direction and the number of pixels H1 in the running direction of the line sensor (image sensor) of the first camera, the number of pixels L2X in the line direction and the number of pixels H2X in the running direction of the line sensor (image sensor) of the second X camera, and the number of pixels L2Y in the line direction and the number of pixels H2Y in the running direction of the line sensor (image sensor) of the second Y camera are as follows: L1: 16000 pixels H1:1 pixel L2X: 2000 pixels H2X: 1 pixel L2Y: 2000 pixels H2Y: 1 pixel

[0112] <Comparative Example> As a comparative example of Numerical Example 1-3, a case where the first camera and the second camera have the same specifications (same standard), that is, a case where the first camera and the second camera have the same pixel size, will be described. In this comparative example, both the first camera and the second camera have 16000 pixels and a pixel size of 5 μm. In this comparative example, the ratio of the imaging width of the second slope imaging region to the imaging width of the first slope imaging region is too small, so that it is insufficient in terms of ensuring an appropriate amount of exposure and the exposure time required to obtain it.

[0113] The imaging parameters for the first and second cameras in the comparative example are as follows: Distance D1 from the imaging reference position to the first slope imaging area: 8.0 m Distance D2 from the imaging reference position to the second slope imaging area: 2.5 m Imaging width Y1 of the first slope imaging area: 1.1 mm Imaging width Y2 of the second slope imaging area: 0.4 mm Pixel size Y1' of the first sensor of the first camera: 5.0 μm Pixel size Y2' of the second sensor of the second camera: 5.0 μm The focal length of the first imaging lens of the first camera is f1: 35.0 mm Imaging angle of the first imaging lens of the first camera θ1: 97.6° The focal length of the second imaging lens of the second camera is f2: 35.0 mm Imaging angle of the second imaging lens of the second camera θ2: 97.6° First camera imaging interval: 0.00014 seconds per pixel Second camera imaging interval: 0.00004 seconds per pixel

[0114] When the resolution is defined as the number of pixels, the number of pixels L1 in the line direction and the number of pixels H1 in the running direction of the line sensor (image sensor) of the first camera, and the number of pixels L2 in the line direction and the number of pixels H2 in the running direction of the line sensor (image sensor) of the second camera are as follows: L1: 16000 pixels H1:1 pixel L2: 16000 pixels H2:1 pixel

[0115] Here, with reference to the above-mentioned numerical examples 1-3 and comparative examples (using them as a reference), it is specified what range of the second imaging width captured by the second imaging unit (second camera) relative to the first imaging width captured by the first imaging unit (first camera) is necessary to ensure an appropriate amount of exposure and the exposure time required to obtain it.

[0116] In Numerical Examples 1 to 3 and Comparative Example, the imaging width of the first slope imaging region by the first imaging unit (first camera) and the imaging width of the second slope imaging region by the second imaging unit (second camera) are as follows: In addition, the ratio (%) of the imaging width of the second slope imaging region by the second imaging unit (second camera) to the imaging width of the first slope imaging region by the first imaging unit (first camera) is also shown. <Numerical Example 1> Imaging width Y1 of the first slope imaging area: 1.1 mm Imaging width Y2 of the second slope imaging area: 1.1 mm Percentage: 100% <Numerical Example 2> Imaging width Y1 of the first slope imaging area: 1.1 mm Imaging width Y2 of the second slope imaging area: 0.8 mm Percentage: 72% <Numerical Example 3> Imaging width Y1 of the first slope imaging area: 2.0 mm Imaging width of 2nd X-side slope imaging area Y2X: 1.9 mm Imaging width of 2nd Y slope imaging area Y2Y: 1.1 mm Percentage (Y1 and Y2X): 95% Percentage (Y1 and Y2Y): 55% <Comparative Example> Imaging width Y1 of the first slope imaging area: 1.1 mm Imaging width Y2 of the second slope imaging area: 0.4 mm Percentage: 36%

[0117] As described above, the ratio (%) of the imaging width of the second slope imaging region by the second imaging unit (second camera) to the imaging width of the first slope imaging region by the first imaging unit (first camera) is 100% in Numerical Example 1, 72% in Numerical Example 2, and 95% or 55% in Numerical Example 3. That is, in this embodiment, the ratio of the imaging width of the second slope imaging region to the imaging width of the first slope imaging region does not have to be 100%, and as long as it is in the range of 50% to 100%, the purpose of inspection in a normal imaging environment can be satisfied. Moreover, among 50% to 100%, it is preferable that it is in the range of 70% to 100%. That is, numerical example 1 illustrates a case where the ratio of the imaging width of the second slope imaging area to the imaging width of the first slope imaging area is 100%, and numerical examples 2 and 3 illustrate cases where the ratio of the imaging width of the second slope imaging area to the imaging width of the first slope imaging area does not necessarily have to be 100% (in the range of 50% to 100%).

[0118] For example, in numerical example 3, the ratio of the imaging width Y2X of the second slope imaging area to the imaging width Y1 of the first slope imaging area is 95%, and the ratio of the imaging width Y2Y of the second slope imaging area to the imaging width Y1 of the first slope imaging area is 55%, which are within the range of 50% to 100%, so that an appropriate amount of exposure and the exposure time required to obtain it can be ensured.

[0119] On the other hand, in the comparative example, the ratio of the imaging width of the second slope imaging region to the imaging width of the first slope imaging region is 36%, which is insufficient in terms of ensuring an appropriate amount of exposure and the exposure time required to obtain it.

[0120] Thus, according to Numerical Example 1-3, which satisfies the range of 50% to 100% of the imaging width of the second slope imaging area by the second imaging unit (second camera) to the imaging width of the first slope imaging area by the first imaging unit (first camera), it is possible to ensure an appropriate amount of exposure and an exposure time to obtain it. On the other hand, in the Comparative Example, which does not satisfy the range of 50% to 100% of the imaging width of the second slope imaging area by the second imaging unit (second camera), it is difficult to ensure an appropriate amount of exposure and an exposure time to obtain it.

[0121] In the imaging system and imaging method of the present embodiment, an imaging unit captures a slope imaging area of ​​an object by dividing the area into a plurality of imaging areas along the moving direction of a moving body. The imaging system and imaging method also include a generating step of generating an image of the slope imaging area by stitching together the respective captured images of the respective imaging areas captured by the imaging method described above. The imaging system and imaging method also include an evaluation step of evaluating the captured image of the slope imaging area obtained by stitching together the respective captured images of the respective imaging areas. The imaging system and imaging method also include a display step of displaying the captured image of the slope imaging area obtained by stitching together the respective captured images of the respective imaging areas. Hereinafter, one aspect of these information processing methods will be described with reference to Figs. 18 to 21.

[0122] FIG. 18 is a sequence diagram showing an example of a display process in the state inspection system.

[0123] The sequence between the evaluation device 3 and the data management device 5 will be described below, but the sequences between the data acquisition device 9, the terminal device 1100, the terminal device 1200 and the data management device 5 are also similar.

[0124] When the user of the evaluation device 3 specifies a folder, the reception unit of the evaluation device 3 receives the selection of the target data (step S91). Alternatively, the user of the evaluation device 3 may select an arbitrary position in map information managed by the map data management unit of the evaluation device 3, and the reception unit of the evaluation device 3 may receive the selection of position information in the map information.

[0125] Next, the communication unit of the evaluation device 3 transmits a request for an input / output screen related to the target data selected in step S91 to the data management device 5, and the communication unit of the data management device 5 receives the request transmitted from the evaluation device 3 (step S92). This request includes the folder name selected in step S91. Alternatively, this request may include location information in map information.

[0126] Next, the storage / reading unit of the data management device 5 searches the processed data management DB 5003 using the folder name included in the request received in step S92 as a search key to read out image data associated with the folder name included in the request. Alternatively, the storage / reading unit searches the acquired data management DB using the location information included in the request received in step S92 as a search key to read out image data associated with the location information included in the request.

[0127] The generating unit of the data management device 5 generates an input / output screen including the image data based on the image data read by the storing / reading unit (step S93). This input / output screen is a screen that accepts an instruction operation to generate an image showing a specific position in the brightness image showing the slope.

[0128] The communication unit of the data management device 5 transmits input / output screen information related to the input / output screen generated in step S93 to the evaluation device 3, and the communication unit of the evaluation device 3 receives the input / output screen information transmitted from the data management device 5 (step S94). Step S94 is an example of a generation reception screen transmission step.

[0129] Next, the display control unit of the evaluation device 3 displays the input / output screen received in step S94 on the display (step S95). The reception unit of the evaluation device 3 receives a predetermined input operation by the user on the displayed input / output screen. This input operation includes an instruction operation to generate an image showing a specific position in the luminance image showing the slope. Step S95 is an example of a reception step.

[0130] The communication unit of the evaluation device 3 transmits input information related to the input operation received by the reception unit to the data management device 5, and the communication unit of the data management device 5 receives the input information transmitted from the evaluation device 3 (step S96). This input information includes instruction information for instructing to generate an image showing a specific position in the luminance image showing the slope.

[0131] The generating unit of the data management device 5 generates a display image using the image data read by the storage / read unit in step S93 based on the received input information (step S97). This display image includes a surface display image including a surface image showing the surface of the slope and a surface position image showing a specific position in the surface image, and a cross-section display image including a cross-section image showing the cross-section of the slope and a cross-section position image showing a specific position in the cross-section image. Step S97 is an example of an image generating step.

[0132] The communication unit of the data management device 5 transmits the display image generated in step S97 to the evaluation device 3, and the communication unit of the evaluation device 3 receives the display image transmitted from the data management device 5 (step S98). Step S98 is an example of a display image transmitting step.

[0133] The display control unit of the evaluation device 3 causes the display image received in step S98 to be displayed on the display (step S99). Step S99 is an example of a display step.

[0134] FIG. 18 shows a sequence relating to the display process between the evaluation device 3 and the data management device 5, but the evaluation device 3 may execute the display process independently.

[0135] In this case, steps S92, 94, 96, and 98 relating to data transmission and reception are omitted, and the evaluation device 3 can perform the same display processing as in Fig. 18 by independently executing steps S91, 93, 95, 97, and 99. The data acquisition device 9, the terminal device 1100, and the terminal device 1200 can each independently execute display processing like the evaluation device 3.

[0136] - Generation of surface display images based on operations specifying specific positions Fig. 19 is an explanatory diagram of operations on the display screen of the state inspection system. Fig. 19 shows an input / output screen 2000 displayed on the display of the evaluation device 3 in step S95 of the sequence diagram shown in Fig. 18, but the same is true for the input / output screen 2000 displayed on each display of the data acquisition device 9, the terminal device 1100, and the terminal device 1200.

[0137] The display control unit of the evaluation device 3 displays an input / output screen 2000 including a designation reception screen 2010 that accepts a designation operation for designating a specific position in a luminance image showing a slope, and a generation reception screen 2020 that accepts an instruction operation for instructing the generation of an image showing a specific position on the slope.

[0138] The display control unit causes a surface image 2100 showing the surface of the slope to be displayed on the designation receiving screen 2010, and also causes a pointer 2300 operated by a pointing device to be displayed on the surface image 2100.

[0139] The surface image 2100 is a luminance image read out from the captured image data in step S92 of FIG. 18, and the display control unit displays the surface image 2100 in correspondence with the captured images 1 and 2 shown in FIG. 5 and the X-axis direction and Y-axis direction shown in the captured image data 7A shown in FIG. 6(a).

[0140] The display control unit displays a generation reception screen 2020 including a specified position confirmation button 2400, a deformation confirmation button 2410, a deformation sign confirmation button 2420, a front view analysis button 2430, a front view comparison button 2440, a cross-sectional view analysis button 2450, and a cross-sectional view comparison button 2460. The deformation confirmation button 2410, the deformation sign confirmation button 2420, the front view analysis button 2430, the front view comparison button 2440, the cross-sectional view analysis button 2450, and the cross-sectional view comparison button 2460 are buttons for instructing to generate an image showing a specific position on the slope, with the position of a part in the surface image 2100 or the cross-sectional image 2200 that satisfies a predetermined condition as the specific position.

[0141] The designated position confirmation button 2400 is a button for confirming a specific position on the slope specified on the designation acceptance screen 2010 and instructing to generate an image showing the specific position on the slope. The designated position confirmation button 2400 may determine not only a specific position specified on the designation acceptance screen 2010, but also a specific position specified by a determination unit or the like and displayed on the designation acceptance screen 2010.

[0142] The deformation confirmation button 2410 is a button that instructs the generation of an image showing a specific position on the slope, with a position indicating a deformation of the slope set as the specific position, and the deformation sign confirmation button 2420 is a button that instructs the generation of an image showing a specific position on the slope, with a position indicating a deformation of the slope set as the specific position.

[0143] The front view analysis button 2430 is a button that instructs the system to generate an image showing a specific position on the slope by specifying a portion obtained by analyzing the surface image 2100 as the specific position, and the front view comparison button 2440 is a button that instructs the system to generate an image showing a specific position on the slope by specifying a portion obtained by comparing the surface image 2100 with another image as the specific position.

[0144] The cross-sectional view analysis button 2450 is a button that instructs the system to generate an image showing a specific position on the slope by specifying a portion obtained by analyzing the cross-sectional image described below as the specific position, and the cross-sectional view comparison button 2460 is a button that instructs the system to generate an image showing a specific position on the slope by specifying a portion obtained by comparing the cross-sectional image with another image as the specific position.

[0145] Fig. 20 is a flow chart showing an example of the process of the information processing method, in which Fig. 20(a) shows the process in the evaluation device 3, and Fig. 20(b) shows the process in the data management device 5.

[0146] When a predetermined position on the surface image 2100 is pointed to by the pointer 2300, the reception unit of the evaluation device 3 receives the pointing operation (step S101), and when the designated position confirmation button 2400 is operated, receives the operation (step S102).

[0147] Next, the determination unit of the evaluation device 3 detects the XY coordinates of the pointed position in the surface image 2100 as a specific position (step S103). This specific position may indicate a point in the XY coordinates, or may indicate an area.

[0148] Next, the communication unit of the evaluation device 3 transmits input information related to the input operation received by the reception unit 32 to the data management device 5 (step S104). This input information includes designation information for designating a specific position in XY coordinates based on a pointing operation using the pointer 2300, and instruction information for instructing to generate an image showing the specific position on the slope based on an operation of the designated position confirmation button 2400.

[0149] The communication unit of the data management device 5 receives the input information transmitted from the evaluation device 3, and the generation unit generates a surface position image that overlaps with the XY coordinates of the specific position by superimposing it on the surface image using the image data shown in Fig. 6(a) based on the instruction information and designation information included in the received input information, thereby generating a surface display image (step S105). The surface position image does not necessarily have to completely match the XY coordinates of the specific position, but only needs to overlap with the XY coordinates of the specific position.

[0150] Next, the generating unit generates a cross-sectional image corresponding to the X-coordinate of the specific position using the image data shown in Fig. 6(a) and the distance measurement data shown in Fig. 6(b) (step S106). If the distance measurement data shown in Fig. 6(b) does not include the X-coordinate of the specific position, the generating unit generates a cross-sectional image based on data in the vicinity of the X-coordinate of the specific position included in the distance measurement data shown in Fig. 6(b).

[0151] In step S106, the generation unit generates a cross-sectional image of a cross-section including the Z-axis direction and the vertical direction shown in FIG. 5, but may generate a cross-sectional image of a cross-section including a direction inclined from the Z-axis direction and the vertical direction, or a cross-sectional image of a cross-section including a direction inclined from the Z-axis direction.

[0152] The generating unit generates a cross-sectional position image overlapping with the Y coordinate of the specific position by superimposing it on the edge line of the cross-sectional image, and generates a cross-sectional display image (step S107).

[0153] The communication unit transmits the surface display image generated in step S105 and the cross-section display image generated in step S107 to the evaluation device 3 (step S108).

[0154] Then, as shown in steps S98 and S99 of FIG. 18, the communication unit of the evaluation device 3 receives the surface display image and the cross-sectional display image transmitted from the data management device 5, and the display control unit of the evaluation device 3 displays the received surface display image and cross-sectional display image on the display.

[0155] Fig. 21 is an example of a display screen after the processing shown in Fig. 18. Fig. 21 shows an input / output screen 2000 displayed on the display of the evaluation device 3 in step S99 of the sequence diagram shown in Fig. 18.

[0156] The display contents of the generation reception screen 2020 are the same as those in FIG. 19, but the display contents of the designation reception screen 2010 are different from those in FIG.

[0157] The display control unit of the evaluation device 3 displays, on the specification reception screen 2010, a surface display image 2150 including a surface image 2100 showing the surface of the slope and a surface position image 2110 showing a specific position in the surface image 2100, and a cross-sectional display image 2250 including a cross-sectional image 2200 showing a cross-section of the slope and a cross-sectional position image 2210 showing a specific position in the cross-sectional image 2200.

[0158] The display control unit causes the cross-sectional image 2200 to be displayed in association with the Y-axis direction and the Z-axis direction shown in FIG.

[0159] Thus, the imaging system of this embodiment has a first imaging unit and a second imaging unit. The imaging system is installed on a moving device and continuously images an object while moving in a moving direction. The first imaging unit images a first imaging area which is a part of the object, and the second imaging unit images a second imaging area which is a part of the object. The second imaging area is closer to the imaging reference position of the first imaging unit and the second imaging unit in a direction intersecting with the moving direction than the first imaging area. The imaging width which is the imaging range in the moving direction when the first imaging unit images the first imaging area in one imaging is defined as the first imaging width, and the imaging width which is the imaging range in the moving direction when the second imaging unit images the second imaging area in one imaging is defined as the second imaging width. In addition, the first imaging parameter for determining the first imaging width in the first imaging unit and the second imaging parameter for determining the second imaging width in the second imaging unit are different so that the second imaging width is within a predetermined range of the first imaging width, which makes it possible to appropriately image areas of the object having different imaging distances.

[0160] Although the invention according to the present disclosure has been described in detail above, it is clear to those skilled in the art that the invention according to the present disclosure is not limited to the embodiments described in the present disclosure. The invention according to the present disclosure can be implemented as modified and altered forms without departing from the spirit and scope of the invention defined based on the description of the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the invention according to the present disclosure.

[0161] The invention as described in the claims of the original application is set forth below. [Appendix 1] An imaging system having a first imaging unit and a second imaging unit, the imaging system is installed on a moving device and captures continuous images of an object while moving in a moving direction; The first imaging unit captures an image of a first imaging region that is a part of the object, The second imaging unit captures an image of a second imaging area that is a part of the object, the second imaging region is closer to the imaging reference position of the first imaging unit and the second imaging unit in a direction intersecting the movement direction than the first imaging region; When an imaging width that is an imaging range in the movement direction when the first imaging unit images the first imaging area in one imaging is defined as a first imaging width, and an imaging width that is an imaging range in the movement direction when the second imaging unit images the second imaging area in one imaging is defined as a second imaging width, A first imaging parameter for determining the first imaging width in the first imaging unit and a second imaging parameter for determining the second imaging width in the second imaging unit are different from each other so that the second imaging width is within a predetermined range of the first imaging width. An imaging system comprising: [Appendix 2] the first imaging parameter is a size of one pixel in the movement direction of a first imaging sensor included in the first imaging unit, the second imaging parameter is a size of one pixel in the movement direction of a second imaging sensor of the second imaging unit, a size of one pixel of the second image sensor in the moving direction is made larger than a size of one pixel of the first image sensor in the moving direction, thereby making the second image capturing width within a predetermined range of the first image capturing width. 2. The imaging system according to claim 1, [Appendix 3] The predetermined range is such that the second imaging width is within a range of 50% to 100% of the first imaging width. 3. The imaging system according to claim 1 or 2, [Appendix 4] The first imaging unit and the second imaging unit are arranged in a line along the moving direction, or in a line along a height direction of the moving device that intersects with the moving direction. 4. The imaging system according to claim 1, wherein the imaging system comprises: [Appendix 5] The first and second imaging areas are disposed at different positions in the moving direction and in the height direction of the object. 5. The imaging system according to claim 1, wherein the imaging system comprises: [Appendix 6] The object is a slope, The moving device is a vehicle. 6. The imaging system according to claim 1, [Appendix 7] An imaging method using an imaging system having a first imaging unit and a second imaging unit, the imaging system is installed on a moving device and captures continuous images of an object while moving in a moving direction; The first imaging unit captures an image of a first imaging region that is a part of the object, The second imaging unit captures an image of a second imaging area that is a part of the object, the second imaging region is closer to the imaging reference position of the first imaging unit and the second imaging unit in a direction intersecting the movement direction than the first imaging region; When an imaging width that is an imaging range in the movement direction when the first imaging unit images the first imaging area in one imaging is defined as a first imaging width, and an imaging width that is an imaging range in the movement direction when the second imaging unit images the second imaging area in one imaging is defined as a second imaging width, A first imaging parameter for determining the first imaging width in the first imaging unit and a second imaging parameter for determining the second imaging width in the second imaging unit are made different from each other, so that the second imaging width is within a predetermined range of the first imaging width. An imaging method comprising: [Explanation of symbols]

[0162] 1. Condition inspection system (imaging system) 6. Mobile objects (vehicles, mobile devices) 7 Imaging System 71 First camera (first imaging unit) 72 Second camera (second imaging unit) 72X 2X camera (2X imaging unit) 72Y 2nd Y camera (2nd Y imaging unit) 60 Mobile Systems

Claims

1. An imaging system having a first imaging unit and a second imaging unit, the imaging system is installed on a moving device and captures continuous images of an object while moving in a moving direction; The first imaging unit captures an image of a first imaging area that is a part of the object, The second imaging unit captures an image of a second imaging area that is a part of the object, the second imaging region is closer to the imaging reference position of the first imaging unit and the second imaging unit in a direction intersecting the movement direction than the first imaging region; When an imaging width that is an imaging range in the movement direction when the first imaging unit images the first imaging area in one imaging is defined as a first imaging width, and an imaging width that is an imaging range in the movement direction when the second imaging unit images the second imaging area in one imaging is defined as a second imaging width, A first imaging parameter for determining the first imaging width in the first imaging unit and a second imaging parameter for determining the second imaging width in the second imaging unit are different from each other so that the second imaging width is within a predetermined range of the first imaging width. An imaging system comprising:

2. the first imaging parameter is a size of one pixel in the movement direction of a first imaging sensor included in the first imaging unit, the second imaging parameter is a size of one pixel in the movement direction of a second image sensor included in the second imaging unit, a size of one pixel of the second image sensor in the moving direction is made larger than a size of one pixel of the first image sensor in the moving direction, so that the second image capturing width is within a predetermined range of the first image capturing width.

2. The imaging system according to claim 1,

3. The predetermined range is a range in which the second imaging width is 50% to 100% of the first imaging width.

3. The imaging system according to claim 1, wherein:

4. The first imaging unit and the second imaging unit are arranged in a line along the moving direction, or in a line along a height direction of the moving device that intersects with the moving direction.

3. The imaging system according to claim 1, wherein:

5. The first and second imaging regions are disposed at different positions in the moving direction and in the height direction of the object.

3. The imaging system according to claim 1, wherein:

6. The object is a slope, The moving device is a vehicle.

3. The imaging system according to claim 1, wherein:

7. An imaging method using an imaging system having a first imaging unit and a second imaging unit, the imaging system is installed on a moving device and captures continuous images of an object while moving in a moving direction; The first imaging unit captures an image of a first imaging area that is a part of the object, The second imaging unit captures an image of a second imaging area that is a part of the object, the second imaging region is closer to the imaging reference position of the first imaging unit and the second imaging unit in a direction intersecting the movement direction than the first imaging region; When an imaging width that is an imaging range in the movement direction when the first imaging unit images the first imaging area in one imaging is defined as a first imaging width, and an imaging width that is an imaging range in the movement direction when the second imaging unit images the second imaging area in one imaging is defined as a second imaging width, A first imaging parameter for determining the first imaging width in the first imaging unit and a second imaging parameter for determining the second imaging width in the second imaging unit are made different from each other, so that the second imaging width is within a predetermined range of the first imaging width. An imaging method comprising:

Images