Information processing device, information processing method, and program

Abstract

To increase an efficiency of forming a reporting document regarding a change in a photographing target in an information processing device that the photographing target is captured in a vehicle equipped with a camera while the vehicle is traveling on a road in a photographing area, and information processing is performed on the captured image.SOLUTION: An information processing device where a photographing target is captured in a vehicle equipped with a camera while the vehicle is traveling on a road in a photographing area, and generates measurement data of the photographing target, comprises: a determination unit that determines whether a photographing completion section photographed by the vehicle while the vehicle is traveling coincides with a separating section that separates the photographing section photographing associated with the photographing target; and a generation unit that generates measurement data of the photographing target corresponding to the photographing completion section by associating it with the separating section based on the determination in the determination unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is a photography system in which a vehicle equipped with a camera travels along a road in a measurement area and photographs a subject to be photographed. For example, a technology is known in which a camera is mounted on a vehicle and photographs the road surface while traveling along the road (see, for example, Patent Document 1).

[0003] Also known is a technology for displaying road segments stored in correspondence with road inspection data on an electronic map as figures with length and width, accepting the selection of a segment, reading the corresponding road inspection data from a memory unit based on the accepted segment selection signal, and outputting it as report data arranged by segment in the order of the extension direction of the target road (see, for example, Patent Document 2). Summary of the Invention [Problem to be solved by the invention]

[0004] There is a demand for a technology for instructing photography to efficiently create a report using measurements (road inspection data) for inspecting road conditions. Note that Patent Documents 1 and 2 do not disclose photography for efficiently creating a report.

[0005] One embodiment of the present invention has been made in consideration of the above problems, and is an information processing device that processes information on images of objects photographed while a vehicle equipped with a camera is driving along roads within a measurement area, and generates measurement data that increases the efficiency of creating report documents regarding changes in the objects photographed. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to one embodiment is an information processing device that captures an image of a photographed object while a vehicle equipped with a camera travels on a road within a measurement area and generates measurement data of the photographed object, the information processing device including: a determination unit that determines whether a photographed section captured by the vehicle while traveling matches a divided section that divides the photographed section associated with the photographed object; and a generating unit that generates measurement data of the object to be photographed corresponding to the photographing-completed section in association with the divided section based on the determination by the determining unit. [Effects of the Invention]

[0007] According to one embodiment of the present invention, an information processing device that processes information on images of an object photographed while a vehicle equipped with a camera is driving along roads within a measurement area can generate measurement data that improves the efficiency of creating report documents regarding changes in the object. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of an imaging system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of a functional configuration of an imaging system according to an embodiment. [Figure 4] 10 is a flowchart illustrating an example of a photographing process according to an embodiment. [Figure 5] 1 is a flowchart (1) illustrating an example of post-photography processing according to an embodiment. [Figure 6] 10 is a flowchart (2) illustrating an example of post-photography processing according to an embodiment. [Figure 7] FIG. 1 is a diagram (1) for explaining measurement area information according to an embodiment. [Figure 8] FIG. 10 is a diagram (2) for explaining measurement area information according to an embodiment. [Figure 9]FIG. 1 is a diagram (1) showing an example of a display screen during post-photography processing according to an embodiment. [Figure 10] FIG. 10 is a diagram (2) showing an example of a display screen during post-photography processing according to an embodiment. [Figure 11] FIG. 3 is a diagram showing an example of a display screen during post-photography processing according to an embodiment. [Figure 12] FIG. 4 is a diagram (4) showing an example of a display screen during post-photography processing according to an embodiment. [Figure 13] FIG. 5 is a diagram (5) showing an example of a display screen during post-photography processing according to an embodiment. [Figure 14] FIG. 6 is a diagram (6) showing an example of a display screen during post-photography processing according to an embodiment. [Figure 15] FIG. 7 is a diagram (7) showing an example of a display screen during post-photography processing according to an embodiment. [Figure 16] FIG. 10 is a diagram illustrating an example of a report record according to an embodiment. [Figure 17] FIG. 10 is a diagram illustrating an example of a state in which an image is captured by a vehicle according to an embodiment. [Figure 18] 2 is a diagram illustrating an example of the installation positions of a first stereo camera and a second stereo camera mounted on a vehicle according to an embodiment. FIG. [Figure 19] 10 is a diagram for explaining how the photographed data is stitched together in accordance with the traveling direction of the vehicle 10. FIG. [Figure 20] FIG. 1 is a diagram illustrating an example of camera installation according to an embodiment. [Figure 21] 21 is a diagram showing an example of an image captured by the camera illustrated in FIG. 20. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention (the present embodiment) will be described in detail with reference to the drawings.

[0010] <System configuration> 1 is a diagram showing an example of the system configuration of an imaging system including an information processing device according to an embodiment. The imaging system 1 includes, as an example, a camera 11, a positioning device 12, an IMU (Inertial Measurement Unit) 13, and an in-vehicle device 100 mounted on a vehicle 10 such as an automobile, and an information processing device 110 capable of communicating with the in-vehicle device 100. The information processing device 110 may be mounted on the vehicle 10 or may be provided outside the vehicle 10. The functions of the information processing device 110 may also be included in the in-vehicle device 100.

[0011] The photography system 1 is a system that photographs a photographic target object while driving a road within a measurement area using a vehicle 10 equipped with a camera 11, and outputs measurement data for inspecting changes to the photographic target object. For example, the photography system 1 photographs the road surface (hereinafter referred to as the road surface) as the photographic target object while driving along the road, thereby collecting data that can be used to inspect changes to the road (road surface condition) as the photographic target object, and outputs the measurement data in categories corresponding to a prescribed report form. Note that changes to the road include cracks, ruts, etc.

[0012] The road surface is an example of a photographed object. The photographed object is not limited to the road surface, and may be, for example, various structures around the road, such as guardrails, guard pipes, road signs, convex mirrors, utility poles, or the inner surfaces of tunnels. The photographing system 1 may photograph one or more photographed objects while traveling on a road, and output measurement data used to create a report for inspecting one or more photographed objects using the photographed images. Here, the following description will be given assuming that the photographing system 1 photographs the road surface, which is the photographed object, while traveling on a road, and outputs measurement data required to create a report for use in inspecting the road, which is the photographed object.

[0013] The camera 11 is an imaging device mounted on the vehicle 10 and captures an image of an object to be photographed. For example, the camera 11 captures an image of a road surface under control of the in-vehicle device 100, and outputs the captured image (hereinafter referred to as a captured image) to the in-vehicle device 100.

[0014] Camera 11 may be a single camera or multiple cameras depending on the object to be photographed. Also, camera 11 may be one or more cameras including multiple imaging elements. Also, camera 11 may be a video camera or a digital camera, or may be a stereo camera capable of acquiring depth information of the photographed image.

[0015] Furthermore, the camera 11 may capture either moving images or continuous still images of the object to be photographed. When continuous still images are taken, it is desirable to take a plurality of images in the direction of travel of the vehicle 10 at the time of photographing, with some of the images overlapping.

[0016] The positioning device 12 is a device that uses, for example, a Global Navigation Satellite System (GNSS) system or the like to output coordinate information or the like that indicates the position of the vehicle 10. The IMU 13 is a unit that includes, for example, a three-axis angular velocity sensor, a three-axis acceleration sensor, and the like, and detects three-dimensional inertial motion.

[0017] The on-vehicle device 100 is a computer mounted on the vehicle 10. The on-vehicle device 100 acquires positioning information indicating the current position of the vehicle 10, for example, based on coordinate information acquired from the positioning device 12 and sensor information acquired from the IMU 13. The on-vehicle device 100 also determines an approach route for the vehicle 10 to enter a measurement area and an exit route for the vehicle 10 to exit the measurement area, based on the positioning information indicating the position of the vehicle 10. The on-vehicle device 100 also controls the camera 11 to capture an object to be captured (for example, a road surface) based on the determined approach route and exit route.

[0018] Furthermore, the on-vehicle device 100 displays on the display 206 (see FIG. 2) or the like a photographing trajectory that indicates the portion of the road within the measurement area to be measured where photographing for measurement has been completed.

[0019] The in-vehicle device 100 may acquire the location information and sensor information of the vehicle 10 managed by the vehicle 10 from an ECU or the like provided in the vehicle 10, for example.

[0020] The information processing device 110 generates measurement area information including an entry detection area for detecting entry onto a road within the measurement area to be measured, an exit detection area for detecting exit from the road, and the like, and transmits the information to the in-vehicle device 100. The information processing device 110 also acquires and stores image data of the road surface within the measurement area photographed by the in-vehicle device 100.

[0021] Furthermore, the information processing device 110 transmits to the vehicle-mounted device 100 division boundary information indicating the division boundary position corresponding to the section (divided section) for outputting inspection data used to inspect the condition of the road within the measurement area to be measured.

[0022] (Processing Overview) Based on the positioning information of the vehicle 10, the in-vehicle device 100 starts photographing the object to be photographed by the camera 11 when the vehicle 10 enters the entry detection area included in the measurement area information, and ends photographing when the vehicle 10 exits the measurement area from the exit detection area included in the measurement area information.

[0023] After completing the photographing based on the measurement area information, the on-board device 100 executes post-photography processing to generate measurement data for inspection based on the image of the photographed object acquired by photographing. For example, the post-photography processing may be executed when the photographing processing for the photographed object (road, etc.) corresponding to all the measurement area information transmitted from the information processing device 110 is completed. Alternatively, the post-photography processing may be executed when the photographing processing corresponding to a certain number of measurement area information is completed.

[0024] Preferably, when the vehicle 10 enters the entry detection area and then exits the measurement area from a location other than the exit detection area, the in-vehicle device 100 stops or disables the camera 11 from capturing an image of the subject.

[0025] After the automatic photography process using the vehicle 10 is completed, post-photography processing is performed to manage the correspondence between the division sections specified in the report used for road inspection and the sections to be photographed for the photographed sections where automatic photography was performed.

[0026] In post-photography processing, a photographed section in which the divided section and the photographed section match is determined to be a measurement-completed section, and a photographed section in which the divided section and the photographed section do not match is determined to be an incomplete measurement section.

[0027] For each set target photographing section, the system automatically selects and identifies from the map any incomplete measurement sections where measurements based on the division sections specified in the report have not been completed.Then, the system sets the incomplete measurement section with the identified points as the start and end points as a new target photographing section, and automatically performs photographing operations.

[0028] In addition, the display of the photographed section for which measurement has been completed is erased from the map, making it possible to visually confirm that measurement has been completed.

[0029] Furthermore, the measurement-incomplete section is displayed on the map superimposed on the division boundary information, and the shooting target section for which measurement has not been completed is displayed corresponding to the division boundary. This allows visual confirmation of the section for which measurement has not been completed within the original shooting target section, and makes it easy to confirm the position corresponding to the measurement-incomplete section on the map display.

[0030] Furthermore, data indicating the measurement incomplete section may be output in combination with point data that can determine the range.

[0031] According to this embodiment, by the above processing, in the photography system 1 in which a vehicle 10 equipped with a camera 11 photographs an object while traveling on a road within a measurement area, the photography operation can be automated, and measurement data for inspection can be generated based on the images obtained by the photography operation.

[0032] <Hardware configuration> The in-vehicle device 100 and the information processing device 110 have, for example, the hardware configuration of a computer 200 as shown in Fig. 2. Note that the information processing device 110 may be configured by a plurality of computers 200.

[0033] Fig. 2 is a diagram showing an example of the hardware configuration of a computer according to an embodiment. For example, as shown in Fig. 2, a computer 200 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a HD (Hard Disk) 204, an HDD (Hard Disk Drive) controller 205, a display 206 corresponding to a display unit, an external device connection I / F (Interface) 207, a network I / F 208, a keyboard 209, a pointing device 210, a DVD-RW (Digital Versatile Disk Rewritable) drive 212, a media I / F 214, and a bus line 215.

[0034] Of these, the CPU 201 controls the overall operation of the computer 200 by executing a predetermined computer program. The ROM 202 stores, for example, computer programs used to start up the computer 200, such as startup files, and data. The RAM 203 is used, for example, as a work area for the CPU 201. The HD 204 stores, for example, computer programs such as an OS (Operating System), applications, and device drivers, as well as various data. The HDD controller 205 controls the reading and writing of various data from and to the HD 204 under the control of the CPU 201, for example. The HD 204 and the HDD controller 205 are examples of storage devices.

[0035] The display 206 displays various types of information such as a cursor, a menu, a window, characters, or an image. The display 206 may be provided outside the computer 200. The external device connection I / F 207 is an interface for connecting various external devices such as the camera 11, the positioning device 12, or the IMU 13 to the computer 200. The network I / F 208 is an interface for communicating with other computers by, for example, wireless communication or wired communication.

[0036] The keyboard 209 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 210 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The keyboard 209 and pointing device 210 may be provided outside the computer 200.

[0037] The DVD-RW drive 212 controls reading and writing of various data from and to a DVD-RW 211, which is an example of a removable recording medium. The DVD-RW 211 is not limited to a DVD-RW, and may be any other removable recording medium. The media I / F 214 controls reading and writing (recording) of data from and to a medium 213 such as a flash memory. The bus line 215 includes an address bus, a data bus, and various control signals for electrically connecting the above-mentioned components.

[0038] 2 is an example of the configuration of the computer 200. The computer 200 may have any other configuration as long as it has, for example, a CPU 201, a ROM 202, a RAM 203, a network I / F 208, and the like.

[0039] <Functional configuration> FIG. 3 is a diagram illustrating an example of the functional configuration of an imaging system according to an embodiment.

[0040] (Functional configuration of the in-vehicle device) The in-vehicle device 100 realizes functional components such as a communication unit 301, an information management unit 302, a positioning unit 303, a determination unit 304, an image capture unit 305, an image capture control unit 306, a display control unit 307, an operation reception unit 308, and an image processing unit 309 by having the CPU 201 execute a program stored in a storage medium. Note that at least a part of the above functional components may be realized by hardware.

[0041] The in-vehicle device 100 also implements a storage unit 310 using storage devices such as the HD 204 and the HDD controller 205, and programs executed by the CPU 201, for example.

[0042] The communication unit 301 connects the in-vehicle device 100 to a communication network using, for example, the network I / F 208 or the like, and executes communication processing to communicate with the information processing device 110. The communication unit 301 may also connect the in-vehicle device 100 to the in-vehicle network of the vehicle 10, and further communicate with various ECUs included in the vehicle 10.

[0043] The information management unit 302, for example, acquires measurement area information received by the communication unit 301 from the information processing device 110, and stores and manages the acquired measurement area information in the storage unit 310, etc. Also, the communication unit 301 acquires division boundary information received from the information processing device 110, and stores and manages the acquired division boundary information in the storage unit 310, etc.

[0044] The positioning unit 303 executes a positioning process for measuring the position of the vehicle 10, for example, using the positioning device 12, the IMU 13, etc. For example, the positioning unit 303 calculates positioning information indicating the position of the vehicle 10, using coordinate information indicating the position of the vehicle 10 output by the positioning device 12 and sensor information output by the IMU 13. Note that the positioning unit 303 may calculate the positioning information of the vehicle 10 by further using driving data acquired from an ECU included in the vehicle 10, etc.

[0045] As another example, the in-vehicle device 100 may acquire the positioning information of the vehicle 10 managed by the vehicle 10 from an ECU provided in the vehicle 10.

[0046] The determination unit 304 determines the entry route for the vehicle 10 to enter the measurement area and the exit route for the vehicle 10 to exit the measurement area based on positioning information indicating the position of the vehicle 10 (hereinafter referred to as the positioning information of the vehicle 10).

[0047] For example, as described above, the measurement area information managed by the information management unit 302 includes an entry detection area that detects entry onto a road within the measurement area, an exit detection area that detects exit from the road, etc. Based on the positioning information of the vehicle 10, the determination unit 304 determines whether the vehicle 10 has entered an entry detection area, whether the vehicle 10 has exited an exit detection area, or whether the vehicle 10 has exited the measurement area from anywhere other than the exit detection area.

[0048] Furthermore, the determination unit 304 determines whether a section in which the vehicle 10 has performed a photographing operation within the measurement area (photographed section) overlaps with a divided section based on the divided section information, based on the positioning information of the vehicle 10. Furthermore, in determining whether a photographed section overlaps with a divided section, the determination unit 304 determines corresponding points to be set in order to select a divided section that is a unit for outputting measurement data for inspection.

[0049] For example, in a photographing operation performed in accordance with measurement area information managed by the information management unit 302, photographed section information is generated based on the positioning information of the vehicle 10 as well, and an overlapping section between the divided section information corresponding to the measurement area information and the photographed section information is determined. Then, corresponding points (starting point and ending point) of each divided section included in the overlapping section are determined, and a measurement-completed section included in the photographed section is determined.

[0050] The photographing unit 305 executes a photographing process for photographing an object to be photographed using the camera 11. The photographing control unit 306 executes a photographing control process for controlling the photographing of the object to be photographed by the camera 11, based on the entry route and exit route determined by the determination unit 304. Note that the photographing unit 305 may be included in the photographing control unit 306.

[0051] For example, when the vehicle 10 enters the measurement area from a predetermined entry detection area, the photography control unit 306 starts photographing the object to be photographed by the camera 11. Furthermore, when the vehicle 10 exits the measurement area from a predetermined exit detection area while the camera 11 is photographing the object to be photographed, the photography control unit 306 ends photographing of the object to be photographed by the camera 11. Furthermore, when, for example, the vehicle 10 exits the measurement area from a location other than the predetermined exit detection area while the camera 11 is photographing the object to be photographed, the photography control unit 306 stops or disables photographing of the object to be photographed by the camera 11.

[0052] Furthermore, the imaging control unit 306 controls the imaging operation according to the newly determined entry detection area and exit area based on the measurement-incomplete section identified by determining the overlapping section between the divided section information and the photographed section information. For example, when a comparison between the measurement area information and the divided section information reveals that an imaging operation corresponding to a specific divided section has not been performed, the imaging control unit 306 resets the entry detection area and exit area based on the corresponding division boundary information in order to re-execute the imaging operation for the divided section that has become a measurement-completed section. Then, the imaging operation is performed based on the reset entry detection area and exit area.

[0053] The display control unit 307 executes display control processing to display various display screens on, for example, the display 206 or a display device provided in the vehicle 10. More specifically, the display control unit 307 executes display control processing to display a map display screen in which, for example, a route 501 (described later) that is the target of the photographing operation and that is included in the measurement area information is superimposed on a map 500 (described later). The display control unit 307 also executes display control processing to display, for example, a photographed section display screen in which a photographed section indicating the range in which the photographing operation was actually performed is superimposed on the map 500 along the route 501. The display control unit 307 also executes display control processing to display, for example, a photographed divided section display screen in which, for example, a division boundary based on divided section information that is compared with the corresponding route 501 is superimposed on the photographed section. Furthermore, the display control unit 307 executes a display control process to, for example, erase the display (photographed trajectory) corresponding to the photographed section from the map display screen on which the division boundary is superimposed on the photographed trajectory, and display a remeasurement area display screen on which only the route 501 corresponding to the measurement incomplete section remains.

[0054] The operation reception unit 308 executes an operation reception process for receiving, for example, an input operation by the user on the display screen displayed by the display control unit 307 .

[0055] The image processing unit 309 executes various image processes on, for example, the captured images captured by the imaging unit 305 and the imaging control unit 306. The storage unit 310 stores, for example, various information such as imaging area information, data, or programs.

[0056] (Functional configuration of information processing device) The information processing device 110 realizes functional components such as a communication unit 311 and a generation unit 312 by causing the CPU 201 to execute a program stored in a storage medium. Note that at least a part of the above functional components may be realized by hardware.

[0057] Furthermore, the information processing device 110 realizes a route information storage unit 313, an image capture data storage unit 314, and a division boundary information storage unit 315 by, for example, storage devices such as the HD 204 and the HDD controller 205, and programs executed by the CPU 201. Note that the route information storage unit 313, the image capture data storage unit 314, and the division boundary information storage unit 315 may be realized by a storage server or a cloud service external to the information processing device 110. Furthermore, when the information processing device 110 is configured by multiple computers 200, the route information storage unit 313, the image capture data storage unit 314, and the division boundary information storage unit 315 may each be realized by a different computer 200.

[0058] The communication unit 311 connects the information processing device 110 to a communication network using, for example, the network I / F 208 or the like, and executes communication processing to communicate with the in-vehicle device 100 or the like. For example, the communication unit 301 executes transmission processing to transmit measurement area information generated by the generation unit 312 to the in-vehicle device 100. The communication unit 301 also executes reception processing to store the imaging data received from the in-vehicle device 100 in the imaging data storage unit 314. The communication unit 301 also executes transmission processing to transmit division boundary position information generated by the generation unit 312 to the in-vehicle device 100. Furthermore, the communication unit 301 receives measurement section determination information notified by the determination unit 304 when the measurement target section has been determined for all the photographed sections.

[0059] The generation unit 312 executes a generation process to generate measurement area information including the above-mentioned entry detection area and exit detection area, etc., based on information on the route to be photographed, map information, etc. The information on the route to be photographed may be input by the user or may be stored in advance in the route information storage unit 313.

[0060] The generating unit 312 executes a generating process to generate division boundary information to be superimposed on the shooting trajectory based on information on the division sections for which measurement data is to be output, map information, etc. Note that the information on the start point and end point of the division section may be input by the user via the operation receiving unit 308, or may be stored in advance in the division boundary information storage unit 315. Note that a specific example of the division boundary information generated by the generating unit 312 will be described later.

[0061] Furthermore, the generating unit 312 generates measurement data as inspection data in accordance with a specified format based on the measurement section determination information received from the on-board device 100 and stores the data in the division boundary information storage unit 315 .

[0062] The route information storage unit 313 stores information necessary for the generation unit 312 to generate measurement area information, such as information on the route to be photographed and map information. The photographing data storage unit 314 stores, for example, photographing data received by the communication unit 311 from the in-vehicle device 100 or the like.

[0063] The division boundary information storage unit 315 stores information about the division sections for which inspection data is to be output, and also stores measurement data generated corresponding to the division sections.

[0064] The functional configuration of the imaging system 1 shown in Fig. 3 is an example. For example, in Fig. 3, at least some of the functional configurations possessed by the information processing device 110 may be possessed by the in-vehicle device 100. Furthermore, at least some of the functional configurations possessed by the in-vehicle device 100 may be possessed by the information processing device 110. Furthermore, at least some of the functional configurations possessed by the imaging system 1 may use functions provided by an external cloud service or the like.

[0065] <Processing flow> Next, the processing flow of the information processing method according to this embodiment will be described.

[0066] (Photography processing) 4 is a flowchart showing an example of a series of information processing including an image capture process and post-image capture process according to one embodiment. This process is information processing executed by the image capture system 1, and shows the overall flow of information processing for measuring changes in the image capture target and outputting the measurement data as inspection data in a specified format.

[0067] In step S401, the generation unit 312 generates measurement shadow area information including an entry detection area for detecting entry onto a road within a measurement area to be photographed, an exit detection area for detecting exit from the road, and the like.

[0068] 7 and 8 are diagrams illustrating measurement area information according to an embodiment. The measurement area information includes, for example, information such as a route 501, a route detection area 502, an entry detection area 504, and an exit detection area 506.

[0069] The route 501 corresponds to, for example, a road to be photographed and a section to be photographed, which is represented on the map 500 by an arrow indicating the direction of travel. The route 501 is made up of, for example, a plurality of points indicating latitude and longitude, and straight lines connecting adjacent points. For example, the route information storage unit 313 stores information about the route 501 to be photographed in advance. Alternatively, the information processing device 110 may store, as the route 501, a road to be photographed and a direction of travel input by a user's input operation in the route information storage unit 313.

[0070] The route detection area 502 is, for example, an area within a range of a predetermined distance from the route 501. For example, the generation unit 312 generates the route detection area 502 by setting an expected positioning error (for example, about 10 m to 30 m) or the like as the predetermined distance.

[0071] The entry detection area 504 is, for example, an area within a range of a first distance from the start point 503 of the route 501 (the start point of the road within the measurement area). For example, the generation unit 312 determines the first distance based on an expected positioning error, a delay in positioning determination, and the like, and sets the area within the range of the first distance from the start point 503 of the route 501 as the entry detection area 504.

[0072] The exit detection area 506 is, for example, an area within a range of a second distance from an end point 505 of the route 501 (the end point of a road within the measurement area). For example, the generation unit 312 determines the second distance based on an expected positioning error, a delay in positioning determination, and the like, and sets the area within the range of the second distance from the end point 505 of the route 501 as the exit detection area 506. Here, the first distance and the second distance may be the same distance or different distances.

[0073] Furthermore, the measurement area information generated by the generation unit 312 may include, for example, a plurality of routes 501a, 501b, and 501c as shown in Fig. 6. In this case, the generation unit 312 generates a route detection area 502, an entry detection area 504, and an exit detection area 506 described in Fig. 5 for each of the plurality of routes 501a, 501b, and 501c.

[0074] 4, the description of the flowchart will be continued. In step S402, the determination unit 304 of the in-vehicle device 100 determines whether or not the photographing process for the predetermined route 501 in the measurement area information has been completed. The photographing process of step S401 is continued until the photographing process for the predetermined route 501 has been completed, and after the photographing process for the predetermined route 501 has been completed, the determination unit 304 shifts the process to step S403.

[0075] As already explained, the photographing process in step S401 is a process for automatically performing an operation of photographing an object to be photographed while moving the vehicle 10 along the route 501 included in the measurement area information.

[0076] In step S403, the communication unit 301 receives the division section information from the division boundary information storage unit 315 of the information processing device 110, and then the display control unit 307 performs display processing on the display 206 to display information about the filmed sections and a screen that displays the division section information on the display 206.

[0077] [Embodiment of post-photography processing] Next, an embodiment of post-photography processing executed by the in-vehicle device 100 will be described.

[0078] 5 is a flowchart showing an example of post-photography processing executed after the photographing processing of the object to be photographed according to one embodiment. This processing is executed, for example, after a series of photographing operations corresponding to one of the predetermined routes 501 is completed by the photographing processing of the object to be photographed executed by the in-vehicle device 100 in step S401 of FIG. 4. For example, the post-photography processing is executed when the vehicle 10 performs the photographing operations while moving from the start point 503 to the end point 505 corresponding to a certain route 501 and then stops the photographing operations. Note that the post-photography processing may be executed not only when the vehicle 10 is stopped, but also while the vehicle 10 is moving, from the completion of the photographing processing in step S401 for a certain route 501 until the start of the photographing processing for the next route 501.

[0079] In step S501, the display control unit 307 of the in-vehicle device 100 displays a photographed section 507 as a completed photographing section corresponding to the photographing trajectory, and a division boundary 508 based on divided section information related to the route 501 corresponding to the photographed section 507, at a predetermined position along the photographed section 507.

[0080] 9 shows an example of a map 500 displayed on the display 206 or the like in step S501. The current location display 101, which indicates the current position of the vehicle 10, has reached a position beyond the route 501, which is the section to be photographed and is displayed based on the measurement area information. The photographed section 507 extends from the starting point 503 of the route 501 to the current location display 101.

[0081] In Fig. 9, the route 501 is shown by a dashed line. Also, the photographed section 507 is shown by a broken line. The display format of these is not limited to the format used in Fig. 9 as long as it is possible to distinguish the respective pieces of information. In the following explanation, the route 501 and the photographed section 507 will be shown in the same display format.

[0082] 9, dividing boundaries 508 are displayed superimposed along the route 501. The positions at which the dividing boundaries 508 are superimposed correspond to a range predefined as a section for acquiring measurement data for inspecting the object to be photographed (the road in this embodiment). Note that the "position corresponding to the predefined range" refers to, for example, a position for distinguishing one section (one unit of a divided section) defined in the report described above. The distance between adjacent dividing boundaries 508 (the distance between sections for which measurement data is acquired) is, for example, 200 meters. Therefore, the display control unit 307 displays the dividing boundaries 508 superimposed on the route 501 on the map 500 every 200 meters from the starting point 503, and proceeds to step S502.

[0083] In step S502, the determination unit 304 determines whether there is an overlap between a photographed section 507 displayed as a photographing trajectory corresponding to one route 501 and a division boundary 508 corresponding to the same route 501. This overlap determination is performed by pattern matching between the photographed section 507 and the division boundary 508 corresponding to the same route 501. The determination unit 304 performs a matching process between the photographed section 507 included in one route 501 and all division boundaries 508 included in the same route 501 by pattern matching, and then proceeds to step S503.

[0084] In step S503, the determination unit 304 determines a corresponding point 509 indicating a measurement-completed section from which measurement data can be acquired in the photographed section 507 based on the results of the matching process between the photographed section 507 and the division boundary 508, and proceeds to step S504.

[0085] 10 is a display example of a map 500 in the middle of the matching process being performed in step S504 between a photographed section 507 (dashed line) included in a route 501 (dotted line) and a division boundary 508. Since the matching process is still in progress, a corresponding point 509 indicating the measurement-completed section is not displayed on the map 500 in FIG.

[0086] In step S504, the determination unit 304 determines that the sections corresponding to the adjacent corresponding points 509 of all the determined corresponding points 509 are the already-photographed sections 507. When the determination unit 304 has determined the corresponding points 509 for all the division boundaries 508 included in one path 501 and the sections of all the corresponding points 509 are the already-photographed sections 507, the determination unit 304 proceeds to step S505.

[0087] In step S505, the determination unit 304 determines corresponding points 509 corresponding to the start and end of the photographed section 507 to identify the boundary of the measurement-completed section for the photographed section 507 corresponding to the route 501 being determined. The determination unit 304 then determines adjacent corresponding points 509 that overlap with the photographed section 507 as boundary points, and associates the photographed section 507 between the two boundary points with the divided section specified in the report. The determination unit 304 also determines the corresponding point 509 closest to the initial point in the direction of travel set on the route 501 and located most upstream as the measurement section start point 509a. The determination unit 304 also determines the corresponding point 509 closest to the end point of the photographing operation and located most downstream as the measurement section end point 509b. The display control unit 307 displays the determined measurement section start point 509a and measurement section end point 509b on the map 500, and proceeds to step S506. The measurement section start point 509a and the measurement section end point 509b may be determined by the user selecting any positions on the map 500 as the measurement section start point 509a and the measurement section end point 509b via the operation reception unit 308.

[0088] 11 shows an example in which the corresponding points 509 determined by the determination unit 304, and the identified measurement section start point 509a and measurement section end point 509b are displayed on the map 500 by the display control unit 307. In this state, the inspection data acquired by the photographing operation corresponds to each divided section.

[0089] In step S506, the determination unit 304 determines whether measurement target sections have been determined for all photographed sections that are the target of post-photography processing (whether corresponding points 509 that become measurement section start points 509a and measurement section end points 509b have been determined). If there is a photographed section for which the measurement target section has not been determined, the determination unit 304 shifts the process to step S501, and executes the processes from step S501 onwards for another photographed section. If there is no photographed section for which the measurement target section has not been determined, the determination unit 304 shifts the process to S601.

[0090] In step S601, the display control unit 307 erases the route 501 for which the measurement target section has been confirmed from the map 500, and proceeds to step S602. This completes the acquisition of inspection data corresponding to the route 501 that was displayed as the imaging target section, making it easy to determine that the route has been confirmed as the measurement target section.

[0091] FIG. 12 illustrates an example in which the display control unit 307 erases the route 501, whose measurement section has been determined, from the map 500.

[0092] In step S602, when the measurement target sections for all photographed sections have been determined, the determination unit 304 notifies the information processing device 110 of measurement section determination information via the communication unit 301. After the information processing device 110 receives the measurement section determination information, the communication unit 301 transitions the process to step S603. In step S602, when the measurement target sections for all photographed sections have not been determined, the determination unit 304 determines that there are photographed sections for which photographing operations have not been performed, and therefore the determination unit 304 returns the process to the photographing process of step S401.

[0093] In step S603, based on the notification received by the communication unit 311 of the information processing device 110, the generation unit 312 generates measurement data as inspection data in accordance with a specified format, stores it in the division boundary information storage unit 315, and terminates post-photography processing.

[0094] 5, a description will be given of the processing to be performed when the determination unit 304 determines that not all of the division boundaries 508 corresponding to the route 501 are included in the photographed section 507 between the determined corresponding points 509. In this case, the determination unit 304 proceeds to step S507.

[0095] 13 is another example of a map 500 displayed by the display control unit 307 during the matching process performed in step S504 between a photographed section 507 (broken line) and a division boundary 508 included in the route 501. Because the matching process is still in progress, the corresponding point 509 indicating the measurement-completed section is not displayed on the map 500 in FIG. 13. As is clear from comparing FIG. 13 with FIG. 10, the photographed section 507 stops midway with respect to the route 501 in FIG. 13.

[0096] In step S507, the determination unit 304 narrows down the corresponding points 509 included in the photographed section 507 by pattern matching the photographed section 507 corresponding to the route 501 being determined and the division boundary 508 included in the same route 501, and then proceeds to step S508.

[0097] In step S508, the determination unit 304 determines adjacent corresponding points 509 among the narrowed-down corresponding points as boundary points, and associates the photographed section 507 between the two boundary points with the divided section defined in the report. The determination unit 304 also determines the corresponding point 509 on the initial point side of the direction of travel set in the route 501 as the measured section start point 509a. The determination unit 304 then determines the corresponding point 509 closest to the photographed section 507, and the corresponding point 509 whose distance to the measured section end point 509b is satisfied by the photographed section 507, as the measured section end point 509b. The display control unit 307 then displays the measured section start point 509a and the measured section end point 509b superimposed on the map 500, and proceeds to step S506.

[0098] 14 shows an example in which the corresponding point 509 determined by the determination unit 304, and the identified measurement section start point 509a and measurement section end point 509b are displayed on the map 500 by the display control unit 307. In this state, the divided section corresponding to the inspection data acquired by the photographing operation is confirmed, so that part of the photographed section 507 of the measurement target section is confirmed as a measured section.

[0099] In step S506, the determination unit 304 determines whether the measurement target sections for all photographed sections 507 that are the subject of post-photography processing have been determined (whether the photographed sections 507 corresponding to all sections included in the corresponding points 509 consisting of a pair of measurement section start point 509a and measurement section end point 509b have been determined).

[0100] As shown in Figure 14, when a photographed section 507 included in one route 501 does not include all of the dividing boundaries 508, the section between the measured section end point 509b, which is the most downstream point of the dividing boundary 508 that matches the photographed section 507, and the corresponding measured section start point 509a is determined to be the measured section.

[0101] When the measured section is determined only halfway along the route 501 as shown in FIG. 14, the determination unit 304 sets the dividing boundary 508 determined as the measured section end point 509b on this route 501 as a new starting point 503 and as a new section to be photographed, and proceeds to step S601.

[0102] In step S601, the display control unit 307 erases from the map 500 the portion of the route 501 for which the measurement target section has been confirmed, and proceeds to step S602. This makes it easy to distinguish between sections of the route 501 that have been displayed as the imaging target section, for which inspection data acquisition has been completed and confirmed as the measurement target section, and sections for which inspection data acquisition has not yet been completed.

[0103] FIG. 15 illustrates an example in which the display control unit 307 erases the route 501, whose measurement section has been determined, from the map 500.

[0104] In step S602, since the determination unit 304 has not determined the measurement target section to include all of the division boundaries 508 included in the route 501 being processed, the determination unit 304 executes the photographing process using the newly set starting point 503 and the original ending point 505 in S506.

[0105] When measuring deformation of an object by processing information from images of the object taken by a vehicle equipped with a camera while driving on roads within a measurement area, if the object range is divided at an arbitrary position without considering correspondence with the division sections specified in the report, the process of obtaining measurement data (data showing deformation such as cracks and ruts) from the photographed images to be recorded in the report becomes complicated.

[0106] For example, if a road is impassable due to road construction or on-street parking, the vehicle will need to drive along the road again later and take new photographs. In this case, to match the photographed sections with the sections specified in the report, complex information processing is required, such as extracting the photographed sections from multiple separate image data, merging the image data, and then obtaining data on cracks, ruts, etc.

[0107] According to each embodiment of the present invention, when the photographing object information acquired by the photographing operation does not satisfy the measurement target range, the photographing system 1 controls the measurement range in accordance with the divided sections defined in the report format, thereby making it possible to omit data merging processing when the photographing target section is missing. In other words, measurement data used to create a report reporting on the deformation of the photographed object can be efficiently acquired, thereby improving the efficiency of report creation.

[0108] <Explanation of the report> Here, an example of a format for a report that defines the divided sections used in post-photography processing will be described. Fig. 16 shows a report sheet T1 as an example based on the format defined for a report. The report sheet T1 corresponds to the measurement object according to this embodiment, and is a data format having a table structure that records the characteristics of the road as the measurement object through the photography processing and post-photography processing performed on the measurement object.

[0109] The information in the "Target Name column C11" included in the record sheet T1 corresponds to the measurement area. Therefore, in the record sheet T2 shown in Figure 16, "City Road 000" is the measurement area.

[0110] In addition, the report sheet T1 defines a section C12 corresponding to the divided section information for dividing the measurement area corresponding to the target name column C11, and provides a column for "Road surface condition survey C30" for inputting the measurement results and the calculation results calculated from the measurement results for each distance specified in the section C12.

[0111] In the example of Fig. 16, the "starting point" and "ending point" of section C12 are set in 100 meter increments, so the division boundaries based on the record sheet T1 are set every 100 meters.

[0112] In addition, the road surface condition survey C30 set up for this section C12 is associated with the ``inspection date C31,'' ``inspection method C32,'' measurement values ​​such as ``cracks C33,'' ``rutting C34,'' ``flatness C335,'' and ``number of patchings (locations) C36,'' and calculated values ​​based on the measurement values ​​such as ``longitudinal irregularity (IRI) C37,'' ``MCI value C38,'' and ``MCI formula selection C39.''

[0113] <Example of shooting state by vehicle 10> FIG. 17 shows a schematic diagram of a measurement area being photographed by a vehicle 10 equipped with an on-board device 100. As shown in FIG. 17, for example, an area continuing in the traveling direction of the vehicle 10 is photographed as different frames (photographed images) at different times so as to include an overlapping photographed area (image overlapping portion) of a predetermined width. The image overlapping portion is set to, for example, 10 to 30% of the width of the photographed image. Note that FIG. 17 schematically shows frames photographed at different times by any one of a total of four imaging elements provided in the first stereo camera 400 and the second stereo camera 402.

[0114] More specifically, as shown in FIG. 18 , the first stereo camera 400 and the second stereo camera 402 are arranged side by side behind the vehicle 10 and simultaneously capture images of the entire width of the road from above. The frame rate of the first stereo camera 400 and the second stereo camera 402 is set to, for example, 30 fps. In this case, the distance traveled in the traveling direction of the vehicle 10 in 1 / 30 s is 0.37 m at 40 km / h or 0.46 m at 50 km / h. In this case, if the width of the captured image in the traveling direction of the vehicle 10 is 0.6 m, the image overlapping portion will be 0.23 m or 0.14 m. The frame rate of the first stereo camera 400 and the second stereo camera 402 may be set according to the traveling speed of the vehicle 10.

[0115] FIG. 19 is a diagram for explaining how image data of a photographed object is created by stitching together images in accordance with the traveling direction of the vehicle 10. When the in-vehicle device 100 captures still images of a photographed area, it is not possible to cover the photographed area in a single shot. In this case, it is desirable to create photographed data of the photographed object by stitching together a plurality of photographed images in accordance with the traveling direction of the vehicle 10 at the time of photographing, as shown in FIG. 19, for example. For example, when the vehicle 10 is traveling along a route 501 in the traveling direction, the in-vehicle device 100 captures images of a predetermined range of the road surface with the camera 11 so that some of the images overlap in the traveling direction.

[0116] <Camera 11 Variations> Furthermore, the camera 11 connected to the in-vehicle device 100 is not limited to having two stereo cameras, and three or more stereo cameras may be provided to efficiently capture images of the width direction of the road. Furthermore, one of the stereo cameras may be set to a different height from the road than the other stereo cameras. Furthermore, the in-vehicle device 100 may be provided with a projection device that projects a texture at the timing when the first stereo camera 400 and the second stereo camera 402 capture images.

[0117] Alternatively, a smartphone or small camera may be installed inside the vehicle 10 to take pictures of the road through the front window.

[0118] For example, the camera 11 connected to the in-vehicle device 100 may be a portable information processing terminal 111 such as a smartphone installed inside the vehicle 10 as shown in Fig. 20(a), or an in-vehicle camera 112, which is an example of a small camera, as shown in Fig. 20(b). In this case, the portable information processing terminal 111 or the in-vehicle camera 112 may be installed inside the windshield FG10 of the vehicle 10, and images captured by these may be processed in the in-vehicle device 100.

[0119] 20, the portable information processing terminal 111 and the in-vehicle camera 112 produce an image generated from an image captured of the area ahead of the vehicle 10, such as image G1 shown in FIG. 21. As shown in FIG. 2, the road surface R on which the vehicle 10 travels is located between the curb K1 and the guardrail K2. A sidewalk H1 exists between the curb K1 and the tree K3 located outside it, and a sidewalk H2 exists between the guardrail K2 and the building K4 located outside it. The camera 11 captures images of the road surface R while traveling along this route 501, and the image G1 is then processed as the image to be measured.

[0120] <Supplementary information> Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the functions described above.

[0121] Additionally, the devices described in the examples are merely one of several computing environments for implementing the embodiments disclosed herein. In one embodiment, the information processing device 110 includes multiple computing devices, such as a server cluster, configured to communicate with each other via any type of communication link, including a network, shared memory, etc., and to perform the processes disclosed herein.

[0122] The information processing device 110 may be integrated into one device or may be divided into multiple devices. Furthermore, at least a part of the functional configuration of the in-vehicle device 100 and the information processing device 110 may be executed using an external cloud service or the like.

[0123] <Additional Notes> This specification discloses the following information processing device, information processing method, and program. (Section 1) An information processing device that captures an object to be photographed while driving a vehicle equipped with a camera on a road within a measurement area and generates measurement data of the object to be photographed, a determination unit that determines whether a photographed section photographed while the vehicle is traveling matches a divided section that divides the photographed section associated with the photographing object; a generation unit that generates measurement data of the object to be photographed corresponding to the photographing-completed section in association with the divided section based on the determination by the determination unit; An information processing device having the above. (Section 2) When the generation unit determines that it is not possible to generate the measurement data for all of the divided sections included in the shooting-completed section, the determination unit sets the section for which measurement data has not yet been generated as a new imaging target section; 2. The information processing device according to claim 1. (Section 3) a display control unit that displays a map showing a measurement area in which the vehicle travels; the determination unit determines a boundary point of the divided section that coincides with the shooting-completed section; the display control unit displays the boundary points superimposed on the map; 3. The information processing device according to claim 1 or 2. (Section 4) A computer that photographs an object to be photographed while driving a vehicle equipped with a camera on a road within a measurement area and generates measurement data of the object to be photographed, a process of determining whether a photographed section photographed while the vehicle is traveling matches a divided section that divides the photographed section associated with the photographed object; generating measurement data of the object to be photographed corresponding to the photographing-completed section in association with the divided section based on the result of the determination; An information processing method that performs the above. (Section 5) A program that causes a computer to execute the photographing method described in item 4.

[0124] Although the embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and various modifications and applications are possible within the scope of the gist of the present invention described in the claims.

[0125] For example, in each of the above embodiments, the vehicle 10 has been described as being an automobile or the like, but the vehicle 10 may be, for example, a train running on a railway. Also, in each of the above embodiments, the in-vehicle device 100 has been described as capturing an image of a subject by combining a plurality of still images, but the in-vehicle device 100 may be configured to capture an image of a subject using moving images, for example. [Explanation of symbols]

[0126] 1: Shooting system 10: Vehicle 11: Camera 12: Positioning device 100: In-vehicle equipment 101: Current location display 110: Information processing device 304: Judgment section 307: Display control unit 308: Operation reception section 310: Storage section 311: Communications Department 312 :Generation part 313: Route information storage unit 314: Shooting data storage unit 315: Division boundary information storage unit [Prior art documents] [Patent documents]

[0127] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-086826 [Patent Document 2] Japanese Patent Application Laid-Open No. 2003-119555

Claims

1. An information processing device that captures an object to be photographed while driving a vehicle equipped with a camera on a road within a measurement area and generates measurement data of the object to be photographed, a determination unit that determines whether a photographed section photographed while the vehicle is traveling matches a divided section that divides the photographed section associated with the photographing object; a generation unit that generates measurement data of the object to be photographed corresponding to the photographing-completed section in association with the divided section based on the determination by the determination unit; An information processing device having the above.

2. When the generation unit determines that it is not possible to generate the measurement data for all of the divided sections included in the shooting-completed section, the determination unit sets the section for which measurement data has not yet been generated as a new imaging target section; The information processing device according to claim 1 .

3. a display control unit that displays a map showing a measurement area in which the vehicle travels; the determination unit determines a boundary point of the divided section that coincides with the shooting-completed section; the display control unit displays the boundary points superimposed on the map; 3. The information processing device according to claim 1 or 2.

4. A computer that photographs an object to be photographed while driving a vehicle equipped with a camera on a road within a measurement area and generates measurement data of the object to be photographed, a process of determining whether a photographed section photographed while the vehicle is traveling matches a divided section that divides the photographed section associated with the photographed object; generating measurement data of the object to be photographed corresponding to the photographing-completed section in association with the divided section based on the result of the determination; An information processing method that performs the above.

5. A program that causes a computer to execute the information processing method according to claim 4.

Images