Information processing device, information processing system, information processing method, and program

Abstract

To provide an information processing device capable of generating a report including an image.SOLUTION: An information processing device provided herein comprises a target object identification unit 322 for identifying a target object included in an image, a slope extension computation unit 323 for computing the size of the target object, and a report generation unit 326 for generating a report on the target, where the report generation unit 326 is configured to cut out an area of the target object from the image and inset the area of the image to a predetermined area of the report to produce a report with a description on the size of the target.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing system, an information processing method, and a program.

Background Art

[0002] From the viewpoint of ensuring traffic safety, it is required to appropriately grasp the state of road geotechnical structures such as slopes. On the other hand, in order to grasp the state of road geotechnical structures, workers had to go to the site and evaluate them.

[0003] Regarding this point, Japanese Patent No. 4551990 (Patent Document 1) discloses a configuration in which a plurality of line cameras are mounted on a vehicle and a panoramic image is created by traveling. According to Patent Document 1, a panoramic video that extends in the moving direction and spreads in the circumferential direction can be created.

[0004] By the way, an administrator (for example, a local government) needs to manage the position, size, shape, type, etc. of road geotechnical structures in order to grasp the state of road geotechnical structures, and may manage them in a report called a document. In some cases, the document may be provided with an area for pasting an image of the road geotechnical structure (hereinafter referred to as a pasting column).

[0005] However, Patent Document 1 does not disclose from the viewpoint of using the created panoramic image in a document.

[0006] Therefore, the development of image processing technology suitable for document creation has been demanded.

Summary of the Invention

Problems to be Solved by the Invention

[0007] The present invention has been made in view of the problems in the above prior art, and an object thereof is to provide an information processing apparatus, an information processing system, an information processing method, and a program for generating a document including an image.

Means for Solving the Problems

[0008] That is, according to the present invention, specific means for identifying an object included in an image, calculating means for calculating the size of the object, creating means for creating a document of the object, are included, the creating means cuts out the area of the object in the image and inserts it into a predetermined area of the document, and provides an information processing apparatus that creates a document with the size of the object described in the document.

Effect of the Invention

[0009] According to the present invention, an information processing apparatus for creating a document including an image can be provided.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0011] Hereinafter, the present invention will be described with reference to embodiments, but the present invention is not limited to the embodiments described below. In the following figures referred to, the same reference numerals are used for common elements, and the description thereof will be omitted as appropriate.

[0012] In the embodiments to be described, a document summarizing information on a geotechnical structure is collectively referred to as a document for reference. However, note that the document may be referred to by various names such as a report or a written report. Also, the document may be printed on paper media or may be displayed on various display devices as electronic data.

[0013] FIG. 1 is a diagram showing a schematic configuration of an information processing system 1 in this embodiment. FIG. 1(a) shows the hardware configuration of the entire information processing system 1, and FIG. 1(b) shows the definition of a normal plane in an exemplary usage state of the imaging system 10.

[0014] In FIG. 1(a), as an example, an environment is illustrated in which a photographing device 110, a server 120, and a terminal device 130 are connected via various networks 140 such as the Internet or a LAN. Note that the number of various devices is not limited to those shown in FIG. 1, and there is no limitation on the number of devices included in the information processing system 1. In the present embodiment, the method of connecting the photographing device 110, the server 120, and the terminal device 130 to the network 140 may be either wired or wireless. Also, the various devices may be directly connected without going through the network 140. In the embodiment to be described, the photographing device 110 is mounted on a moving body such as a vehicle as shown in FIG. 1(a). Here, the moving body on which the photographing device 110 is mounted is referred to as a photographing system 10.

[0015] Here, terms used in the present embodiment are defined. The “road geotechnical structure” is a general term for structures composed mainly of ground materials such as earth, sand, and rock constructed for road construction and structures attached thereto, and refers to cut slopes, slope stabilization facilities, fills, culverts, and the like. Also, artificially processed slopes such as these cut slopes, slope stabilization facilities, fills, culverts, and slopes with an artificial structure are referred to as “slopes”. In other words, the slopes related to the road geotechnical structure are referred to as slopes. Hereinafter, the road geotechnical structure may be referred to as an “object”, a “slope”, or an “object including a slope”.

[0016] The imaging device 110 is a device for imaging road geotechnical structures, and is configured to include a camera, various sensors, and the like. The imaging device 110 of the present embodiment is mounted on a moving body such as a vehicle, and can image the road geotechnical structures provided beside the road by traveling on the road. Further, in the embodiment to be described, the imaging device 110 is described as imaging the slope provided beside the road. The imaging device 110 of the present embodiment can image an image of the side of the road including the slope provided beside the road by imaging while traveling by vehicle. Here, the length of the slope is defined. The width direction of the slope (the X direction in FIG. 1(b)) orthogonal to the height direction of the slope (the Y direction in FIG. 1(b)) when the slope is viewed from the front is defined as the "extension direction". In the slope provided beside the road, the "extension direction" refers to the moving direction of the vehicle moving on the road. Further, in the embodiment to be described, the "extension" of the slope refers to the size of the slope in the width direction of the slope, that is, the length of the slope in the moving direction of the vehicle.

[0017] Further, the imaging device 110 of the present embodiment can capture an image and acquire sensor data related to the imaging situation by various sensors. For example, the imaging device 110 of the present embodiment includes a GPS (Global Positioning System) device and can acquire the position information of the position where the image is captured. Thereby, the starting position and the ending position of the slope included in the image can be specified. Further, the imaging device 110 of the present embodiment includes a distance sensor such as a LiDAR (Light Detection And Ranging) system and can measure the distance to the captured slope.

[0018] Server 120 is an information processing device such as a server computer, for example. The server 120 in this embodiment processes the image data acquired by the imaging device 110 and various sensor data acquired by various sensors to create a document. The document can describe various information such as, for example, an image of a slope, an extension of the slope, and the positions of the starting point and the ending point of the slope. The created document is provided to a local government or a jurisdiction department that manages the road through a network or various recording media. Also, the created document may be printed on a paper medium and provided to an administrator or the like.

[0019] The terminal device 130 is an information processing device such as a personal computer, for example. The terminal device 130 in this embodiment can perform operations related to document creation and image processing. Also, the terminal device 130 can display the created document. Note that the terminal device 130 is not limited to the form of a personal computer as shown in Fig. 1(a), and may be, for example, a tablet terminal or the like.

[0020] The information processing system 1 shown in Fig. 1(a) exemplifies a so-called cloud-configured system in which the server 120 provides a service for creating the document of this embodiment, but does not particularly limit the embodiment. Therefore, for example, a so-called stand-alone configuration in which a process of creating a document is executed on an application of the terminal device 130 may be used.

[0021] Next, the hardware configurations included in various devices will be described. FIG. 2 is a diagram showing the hardware configurations included in each device constituting the information processing system 1 of the present embodiment. FIG. 2(a) shows the hardware configuration of the imaging device 110, and FIG. 2(b) shows the hardware configurations of the server 120 and the terminal device 130. As shown in FIG. 2(a), the imaging device 110 of the present embodiment includes a CPU 201, a RAM 202, a ROM 203, a storage device 204, a communication I / F 205, a camera 206, a GPS device 207, and a distance sensor 208, and each hardware is connected via a bus. Further, as shown in FIG. 2(b), the server 120 and the terminal device 130 of the present embodiment include a CPU 201, a RAM 202, a ROM 203, a storage device 204, a communication I / F 205, a display 209, and an input device 210, and each hardware is connected via a bus.

[0022] First, referring to FIG. 2(a), the hardware of the imaging device 110 will be described. The CPU 201 is a device that executes a program for controlling the operation of the imaging device 110 and performs predetermined processing. The RAM 202 is a volatile storage device for providing an execution space for the program executed by the CPU 201, and is used for storing and expanding programs and data. The ROM 203 is a non-volatile storage device for storing programs and firmware executed by the CPU 201.

[0023] The storage device 204 is a readable and writable non-volatile storage device that stores an OS, various software, setting information, various data, etc. for operating the imaging device 110. Examples of the storage device 204 include an HDD (Hard Disk Drive) and an SSD (Solid State Drive). The storage device 204 of the present embodiment can store, for example, captured image data, position data of the shooting location, distance data to the slope surface, etc.

[0024] The communication I / F 205 connects the imaging device 110 and the network 140, enabling communication with other devices via the network 140. The communication via the network 140 may be either wired communication or wireless communication, and can use a predetermined communication protocol such as TCP / IP to transmit and receive various data.

[0025] The camera 206 is a device that captures images of road geotechnical structures such as slopes, and constitutes the imaging means of this embodiment. The camera 206 of this embodiment may be configured in the form of, for example, a line sensor, and as the vehicle equipped with the imaging device 110 travels on the road, it can capture an image of the slope scanned in the extension direction. Note that the camera 206 does not have to be a line sensor, and may be configured in the form of, for example, an area sensor. Note that the imaging device 110 of this embodiment may be equipped with one camera 206 or may be equipped with a plurality of cameras 206.

[0026] The GPS device 207 is a device that receives radio waves from GPS satellites and acquires data for calculating the latitude and longitude of the vehicle equipped with the imaging device 110 based on the radio waves. The GPS device 207 of this embodiment can constitute the position information acquisition means of this embodiment. Also, the GPS device 207 of this embodiment may be configured to operate in conjunction with the camera 206, and by acquiring data at the timing when the camera 206 captures an image, the location where the image was captured can be specified.

[0027] The distance sensor 208 is a device that measures the distance from the imaging device 110 to the object to be imaged, and constitutes the distance measuring means of the present embodiment. The distance sensor 208 of the present embodiment can be configured, for example, in the form of a LiDAR system. LiDAR is a method of measuring the time of flight of light using pulses. As another method of the ToF sensor, the distance may be measured by the phase difference detection method. In the phase difference detection method, a laser beam amplitude-modulated at the fundamental frequency is irradiated to the measurement range, the reflected light is received, the phase difference between the irradiated light and the reflected light is measured to obtain the time, and the distance is calculated by multiplying the time by the speed of light. Further, the distance sensor 208 may be configured by a stereo camera or the like. Further, the distance sensor 208 of the present embodiment may be configured to operate in conjunction with the camera 206, and by measuring the distance at the timing when the camera 206 captures an image, the distance to the object (for example, the normal plane) included in the captured image can be specified.

[0028] Next, with reference to FIG. 2(b), the hardware of the server 120 and the terminal device 130 will be described. Note that the PU 201, RAM 202, ROM 203, storage device 204, and communication I / F 205 shown in FIG. 2(b) have the same configuration as those shown in FIG. 2(a), and thus the description thereof will be omitted.

[0029] The display 209 is a device that displays various data and the states of the server 120 or the terminal device 130 to the user. Examples include an LCD (Liquid Crystal Display). The display 209 of the present embodiment is configured as a display unit, and can display, for example, a processing screen for creating a document, the created document, and the like. The input device 210 is a device for the user to operate the server 120 or the terminal device 130. Examples include a keyboard and a mouse. Note that the display 209 and the input device 210 may be separate devices, or may be a device having both functions such as a touch panel display. Further, the server 120 may be configured not to include the display 209 and the input device 210.

[0030] The hardware configuration included in each device of the information processing system 1 of the present embodiment has been described above. Next, the functional means executed by each hardware in the present embodiment will be described with reference to FIG. 3. FIG. 3 is a software block diagram included in the information processing system 1 of the present embodiment.

[0031] As shown in FIG. 3, the imaging device 110 of the present embodiment includes functional means such as a communication unit 311, an imaging unit 312, a position information acquisition unit 313, and a distance measurement unit 314. The server 120 of the present embodiment includes functional means such as a communication unit 321, an object identification unit 322, a slope extension calculation unit 323, a divided image generation unit 324, a latitude and longitude calculation unit 325, a document creation unit 326, and a data storage unit 327. The terminal device 130 of the present embodiment includes functional means such as a communication unit 331 and a display control unit 332. Details of each functional means will be described below.

[0032] First, the functional means of the imaging device 110 will be described. The communication unit 311 is a means for controlling the operation of the communication I / F 205 and communicating between the imaging device 110 and the network 140. The communication unit 311 of the present embodiment constitutes a communication means. For example, the communication unit 311 of the present embodiment can transmit image data, data indicating position information, data on the distance from the imaging device 110 to an object, etc. to the server 120 via the network 140.

[0033] The imaging unit 312 constitutes the imaging means of the present embodiment and is a means for capturing an image by controlling the operation of the camera 206. The imaging unit 312 of the present embodiment can capture an image in accordance with the movement of the vehicle on which the imaging device 110 is mounted. Thereby, a panoramic image extending in the moving direction of the vehicle can be captured, and an image that easily grasps the extension direction of the slope can be captured. Note that the captured image data may be transmitted to the server 120 via the communication unit 311 or stored in the storage device 204 of the imaging device 110.

[0034] Here, regarding the image captured by controlling the operation of the imaging unit 312 with the camera 206, an explanation will be given with reference to FIG. 5. FIG. 5 is a diagram for explaining the stitching of images in the present embodiment. In the present embodiment, each of a plurality of images captured in accordance with the movement of the vehicle is defined as a partial image. The imaging unit 312 of the present embodiment outputs a single composite image by stitching together a plurality of partial images captured by controlling the operation of the camera 206 in accordance with the movement of the vehicle. That is, as shown in FIG. 5, the image of the present embodiment is configured as a composite image by stitching together the partial images p1 to pl captured along the moving direction of the vehicle.

[0035] Returning to the explanation with reference to FIG. 3. The position information acquisition unit 313 is a means for acquiring data related to radio waves from GPS satellites received via the GPS device 207, and constitutes the position information acquisition means of the present embodiment. By the position information acquisition unit 313 acquiring position information from a plurality of GPS satellites, the position of the imaging device 110 can be specified. The position information acquisition unit 313 of the present embodiment can acquire position information in accordance with the timing when the imaging unit 312 captures an image. Note that the data related to the acquired position information may be transmitted to the server 120 via the communication unit 311, or may be stored in the storage device 204 of the imaging device 110.

[0036] The distance measurement unit 314 constitutes the measurement means of the present embodiment, and is a means for calculating the distance from the imaging device 110 to the object by controlling the operation of the distance sensor 208 and processing the data output from the sensor. The distance measurement unit 314 of the present embodiment can perform distance measurement and calculation in accordance with the timing when the imaging unit 312 captures an image. Note that the measured distance data may be transmitted to the server 120 via the communication unit 311, or may be stored in the storage device 204 of the imaging device 110.

[0037] Next, the functional means of the server 120 will be described. The communication unit 321 is a means for controlling the operation of the communication I / F 205 and performing communication between the server 120 and the network 140. The communication unit 321 in the present embodiment constitutes a communication means. Further, the communication unit 321 may be configured as an acquisition means for acquiring various data. The communication unit 321 can transmit or receive various data. For example, the communication unit 321 in the present embodiment can receive image data, position information data, distance data, etc. from the imaging device 110. The various data received by the communication unit 321 can be stored in the storage device 204 via, for example, the data storage unit 327. Also, the communication unit 321 in the present embodiment can transmit a document to the terminal device 130.

[0038] The object identification unit 322 constitutes the identification means in the present embodiment and is a means for identifying an object included in an image received from the imaging device 110. The object identification unit 322 in the present embodiment can identify an object (e.g., a slope surface, etc.) included in an image by analyzing the image. Also, the object identification unit 322 in the present embodiment can also identify an object by a user's operation. In this case, the object identification unit 322 can also be configured as a reception means for receiving a user's operation.

[0039] The slope extension calculation unit 323 constitutes the calculation means in the present embodiment and is a means for calculating the size of an object, that is, the extension of the slope surface included in the image. The calculated extension data is output to the divided image generation unit 324, the document creation unit 326, the data storage unit 327, etc. and is processed by various functional means. Here, a method by which the slope extension calculation unit 323 in the present embodiment calculates the extension of the slope surface will be described with reference to FIG. 6.

[0040] FIG. 6 is a diagram for explaining a method of calculating the extension of a slope surface in the present embodiment. FIG. 6(a) shows an example of calculating the extension of a slope surface by an exact formula, and FIG. 6(b) shows an example of calculating the extension of a slope surface by an approximate formula. FIG. 6 shows a lens constituting the optical system of the camera 206 and the configuration of that lens.

[0041] Note that each symbol in FIGS. 6(a) and 6(b) is referenced as follows. That is, in FIG. 6, H1 and H2 respectively indicate the principal point positions on the subject side and the imaging element side. F1 and F2 respectively indicate the focal positions on the subject side and the imaging element side. O indicates the position of the subject. Y’ indicates the pixel size. f indicates the distance (focal length) from the principal point position H1 on the subject side to the focal position F1 on the subject side. x indicates the distance from the focal position F1 on the subject side to the subject. s indicates the distance from the principal point position H1 on the subject side to the subject. Here, if the size Y of one pixel in the subject in the captured image is obtained, the extension of the normal plane can be obtained by multiplying Y by the number of pixels constituting the extension direction of the normal plane.

[0042] First, the example of FIG. 6(a) will be described. In the case of the exact formula, Y can be calculated as in the following formula 1.

[0043]

Equation

[0044] Here, s is obtained as a value from the output of the distance measurement unit 314. Also, f and Y’ are known values determined by the specifications of the camera 206. Therefore, Y can be calculated by the above formula 1, and the normal plane extension calculation unit 323 can calculate the extension of the normal plane by multiplying Y by the number of pixels constituting the extension direction of the normal plane.

[0045] Next, the example of FIG. 6(b) will be described. In the case of the approximate formula, Y can be calculated as in the following formula 2.

[0046]

Equation

[0047] Here, the value of d is obtained from the output of the distance measurement unit 314. Also, f and Y' are known values determined by the specifications of the camera 206. Therefore, Y can be calculated by the above formula 1, and the normal plane extension calculation unit 323 can calculate the extension of the normal plane by multiplying Y by the number of pixels constituting the extension direction of the normal plane.

[0048] Note that the method by which the normal plane extension calculation unit 323 calculates the extension of the normal plane is not limited to the above-described method, and does not particularly limit the embodiment. Therefore, the normal plane extension calculation unit 323 may calculate the normal plane by a method other than that shown in FIG. 6.

[0049] Returning to the description with reference to FIG. 3, the divided image generation unit 324 constitutes the generation means of the present embodiment, and is a means for generating an image obtained by dividing a composite image formed by connecting the photographed partial images p1 to pl (hereinafter referred to as a divided image). The divided image generation unit 324 of the present embodiment generates a divided image when the extension of the normal plane calculated by the normal plane extension calculation unit 323 is longer than a predetermined threshold value. By generating the divided image in this way by the divided image generation unit 324, the visibility of the image of the normal plane inserted into the document can be improved, and the convenience for the viewer of the document can be improved.

[0050] The latitude and longitude calculation unit 325 constitutes the calculation means of the present embodiment, and is a means for calculating the latitude and longitude based on the position information data acquired by the position information acquisition unit 313. The latitude and longitude information obtained by the latitude and longitude calculation unit 325 may be output to the document creation unit 326 or stored in the data storage unit 327.

[0051] The document creation unit 326 constitutes the creation means of the present embodiment, and is a means for creating a document regarding the slope based on various data. The document creation unit 326 of the present embodiment can create a document including an ID for identifying the slope, an image of the entire photographed slope, a divided image, the extension of the slope, and the latitude and longitude of the starting point and ending point of the slope. Further, in another preferred embodiment, the document can include information indicating whether the slope is a cut slope or a fill slope, the height of the slope, information indicating whether the slope corresponds to a so-called specific geotechnical work (road geotechnical structure exceeding a predetermined size), and other remarks. The document creation unit 326 of the present embodiment can create a document by referring to the document creation data management table. The document created by the document creation unit 326 may be displayed on, for example, the display 209, or may be stored in the data storage unit 327. Further, the document may be printed on a paper medium and provided to a road administrator or the like, or may be provided as electronic data via the network 140.

[0052] The data storage unit 327 constitutes the storage means of the present embodiment, and is a means for controlling the operation of the storage device 204 and performing writing and reading of various data. The data storage unit 327 of the present embodiment can store, for example, image data, divided image data, position information data, data indicating latitude and longitude, distance data, and electronic data of documents.

[0053] Further, the data storage unit 327 of the present embodiment may store, for example, in a table format, the data required for creating a document, as a document creation data management table. Here, the document creation data management table of the present embodiment will be described with reference to FIG. 7.

[0054] FIG. 7 is a diagram showing an example of a document creation data management table in the present embodiment. As shown in FIG. 7, for example, the document creation data management table can store by associating a slope ID, a location number, a slope extension, a slope height, a flag indicating whether it corresponds to specific soil engineering, coordinates of a starting point position, coordinates of an ending point position, and a name of a structure. Further, information (such as a path and a file name) indicating image data of each slope can be associated with the data of each slope in the document creation data management table. Note that the configuration of the document creation data management table shown in FIG. 7 is an example, and other items may be stored.

[0055] Returning to FIG. 3 for explanation again, the functional means of the terminal device 130 will be described. The communication unit 331 is a means for controlling the operation of the communication I / F 205 and communicating between the terminal device 130 and the network 140. The communication unit 311 of the present embodiment constitutes a communication means. For example, the communication unit 331 of the present embodiment can receive a document created by the server 120.

[0056] The display control unit 332 constitutes the display control means of the present embodiment, and can display various information by controlling the operation of the display 209. The display control unit 332 of the present embodiment can perform control to display, for example, a captured image, a screen for creating a document, a screen for viewing the created document, etc. on the display 209. Further, the display control unit 332 of the present embodiment can display a setting screen for the starting point and the ending point of the slope. The user can set the starting point and the ending point of the slope by performing an operation of arranging, for example, a marker indicating the starting point and a marker indicating the ending point on the image on the displayed setting screen.

[0057] Note that the above-described software blocks correspond to functional means realized by causing each hardware to function by the CPU 201 executing the program of the present embodiment. Further, all of the functional means shown in each embodiment may be realized software-wise, or a part or all of them may be implemented as hardware providing equivalent functions.

[0058] Furthermore, not all of the above-described functional means necessarily need to be included in each device in the configuration as shown in FIG. 3. For example, in another preferred embodiment, each functional means may be realized by the cooperation of two or more of the devices among the imaging device 110, the server 120, and the terminal device 130. For example, the imaging device 110 may be provided with a functional means corresponding to the latitude and longitude calculation unit 325 in FIG. 3, and may be configured to transmit the calculated latitude and longitude to the server 120 via the communication unit 311. Also, for example, when the service according to the present embodiment is provided in a stand-alone form, the functional means of the server 120 and the terminal device 130 may be included in one information processing device.

[0059] Here, an example of the information processing system 1' configured in a stand-alone form will be described with reference to FIG. 4. FIG. 4 is a software block diagram included in the information processing system 1' of another embodiment.

[0060] As shown in FIG. 4, the information processing system 1' includes an imaging device 110 and an information processing device 150 connected via a network 140. The imaging device 110 includes each functional means of a communication unit 311, an imaging unit 312, a position information acquisition unit 313, and a distance measurement unit 314. Since each functional means included in the imaging device 110 in FIG. 4 is the same as the functional means shown in FIG. 3, detailed description thereof will be omitted.

[0061] Also, the information processing device 150 includes each functional means of a communication unit 351, an object identification unit 352, a normal plane extension calculation unit 353, a divided image generation unit 354, a latitude and longitude calculation unit 355, a document creation unit 356, a data storage unit 357, and a display control unit 358. Each functional means of the communication unit 351, the object identification unit 352, the normal plane extension calculation unit 353, the divided image generation unit 354, the latitude and longitude calculation unit 355, the document creation unit 356, the data storage unit 357, and the display control unit 358 in FIG. 4 respectively corresponds to each functional means of the communication unit 321, the object identification unit 322, the normal plane extension calculation unit 323, the divided image generation unit 324, the latitude and longitude calculation unit 325, the document creation unit 326, the data storage unit 327, and the display control unit 332 shown in FIG. 3, and thus detailed description thereof will be omitted.

[0062] As shown in FIG. 4, the information processing apparatus 150 includes each functional means of the server 120 and the terminal device 130 in FIG. 3, whereby an information processing system 1' that executes processing in a stand-alone form can be realized.

[0063] Next, the processing executed by each of the above-described functional means will be described. FIG. 8 is a flowchart showing the processing of creating a document by the server 120 and the terminal device 130 of the present embodiment. The server 120 starts the processing from step S1000.

[0064] In step S1001, the display control unit 332 of the terminal device 130 displays, on the display 209 of the terminal device 130, a screen for processing the image of the slope surface based on the evaluation data (image data, various sensor data, etc.) received from the server 120. Here, the image processing screen of the present embodiment will be described with reference to FIG. 9. FIG. 9 is a diagram showing an example of the image processing screen of the slope surface in the present embodiment.

[0065] The image processing screen shown in FIG. 9 includes the captured image and various operation buttons and is displayed. The user can perform various image processes and the like by operating the buttons within the image processing screen. Further, the image processing screen of the present embodiment includes a document creation button 902. For example, after the user selects an image to be processed (hereinafter referred to as "processing target image"), when the document creation button 902 is pressed, a document creation request is transmitted from the terminal device 130 to the server 120. Triggered by the server 120 receiving this document creation request, the document creation unit 326 of the server 120 executes the process of creating a document related to the processing target image. The user can select the processing target image by the image switching buttons 901L and 901R in FIG. 9.

[0066] Return to FIG. 8. After step S1001, in step S1002, the process branches depending on whether there is a document creation request. Here, the presence or absence of a document creation request can be determined by whether the document creation button 902 has been pressed on the image processing screen displayed in step S1001. If there is no document creation request in step S1001 (NO), the process returns to step S1001 and waits for a document creation request. On the other hand, if there is a document creation request (YES), the process proceeds to step S1003.

[0067] In step S1003, the object identification unit 322 of the server 120 identifies the object included in the processing target image. Here, the object identified in the described embodiment is a slope. The object identification unit 322 can identify the slope by analyzing the processing target image. Also, the object identification unit 322 of the server 120 may identify the slope based on the user operation performed on the setting screen displayed by the display control unit 332 of the terminal device 130. Here, a method of identifying the slope by the user operation will be described with reference to FIG. 10.

[0068] FIG. 10 is a diagram showing an example of a screen (hereinafter referred to as a setting screen) for setting the start point and end point of a slope in the present embodiment. As shown in FIG. 10, the display control unit 332 performs control to display on the display 209 of the terminal device 130 a screen in which a marker indicating the start point (hereinafter referred to as the start marker Ms) and a marker indicating the end point (hereinafter referred to as the end marker Me) are superimposed on the image.

[0069] As shown in FIG. 10, the display control unit 332 of the terminal device 130 can display a start position designation button 1001, an end position designation button 1002, a specific position determination button 1003, a reduction button 1004, and an enlargement button 1005 in the setting screen. The start position designation button 1001 and the end position designation button 1002 are buttons for instructing to display a start marker Ms and an end marker Me on the image, respectively. The specific position determination button 1003 is a button for determining the positions of the start marker Ms and the end marker Me on the image. The reduction button 1004 and the enlargement button 1005 are buttons for instructing to display the image in a reduced or enlarged manner.

[0070] In the setting screen of FIG. 10, when the user presses the start position designation button 1001, the display control unit 332 displays the start marker Ms at an arbitrary position on the image. Also, in the setting screen of FIG. 10, when the user presses the end position designation button 1002, the display control unit 332 displays the end marker Me at an arbitrary position on the image.

[0071] The user can perform an operation of moving the start marker Ms and the end marker Me in the setting screen shown in FIG. 10 to set the start and end points of the normal plane. The start marker Ms and the end marker Me can be moved to any position on the image by operating the pointer. The user can specify the normal plane by arranging the start marker Ms and the end marker Me at the boundary between the area of the normal plane and the area that is not the normal plane within the image. Here, the operation of moving the start marker Ms and the end marker Me can be performed by drag-and-drop with a mouse, swiping on a touch panel, pressing a direction key, or the like. When the image includes a plurality of normal planes, the object specifying unit 322 can specify the plurality of normal planes by arranging a plurality of start markers Ms and a plurality of end markers Me by the user's operation.

[0072] When the server 120 receives from the terminal device 130 that the user has pressed the specific positioning button 1003, the object specifying unit 322 of the server 120 accepts it as a determination operation to specify a partial area in the image via the communication unit 311 and the communication unit 331. That is, the result (such as coordinate information) of the operation (such as moving a marker) performed by the terminal device 130 on the setting screen shown in FIG. 10 is transmitted to the server 120, and the object specifying unit 322 can specify a normal plane based on the result. Note that the specification of the normal plane is not limited to being performed by image analysis or by the user's operation. Therefore, for example, after performing image analysis, the user may move the marker. Thereby, the user can correct the positions of the start point and the end point of the normal plane specified by image analysis to more appropriately specify the normal plane. Note that the method of specifying the normal plane is not limited to the method using the start marker Ms and the end marker Me at the boundary between the area of the normal plane and the area not being the normal plane in the image. For example, a method of surrounding the area of the normal plane in the image with a frame may be used so that the boundary between the area of the normal plane and the area not being the normal plane in the image becomes clear. Even in this case, the portion indicating the boundary between the area of the normal plane and the area not being the normal plane in the image within the frame is a marker.

[0073] Returning the explanation to FIG. 8. After specifying the normal plane in step S1003, in step S1004, the normal plane extension calculation unit 323 of the server 120 calculates the extension of the normal plane specified in step S1003. As described with reference to FIG. 6, the normal plane extension calculation unit 323 of the present embodiment can calculate the extension of the normal plane based on the size of one pixel of the object in the processing target image and the number of pixels constituting the extension direction of the normal plane.

[0074] After that, in step S1005, the document creation unit 326 of the server 120 cuts out the image of the specified slope portion from the image and inserts it into a predetermined area of the document. Here, an example of a document into which the image of the slope portion is inserted will be described with reference to FIGS. 11 and 12. FIG. 11 is a diagram showing a first example of a document including an image of a slope portion in the present embodiment. FIG. 12 is a diagram showing a second example of a document including an image of a slope portion in the present embodiment.

[0075] As shown in FIGS. 11 and 12, the image of the slope portion is inserted into a column (predetermined area, the column described as slope image in FIGS. 11 and 12) for attaching the slope image in the document. For example, in FIGS. 11 and 12, the column for attaching the slope image is in the lower area of the document, and the size of the object is inserted into the upper area of the document. Here, as shown in FIGS. 11 and 12, the "shooting range" is the "whole" of the slope. In this way, the document creation unit 326 of the server 120 in the present embodiment can create a document including the specified object.

[0076] Returning to the description with reference to FIG. 8. In step S1006, the process branches depending on whether the extension of the slope is greater than a predetermined threshold. If the extension of the slope is smaller than the predetermined threshold (NO), the process proceeds to step S1009. If the extension of the slope is greater than the predetermined threshold (YES), the process proceeds to step S1007 to divide the image.

[0077] Here, an example of generating a divided image in the present embodiment will be described with reference to FIG. 13 in addition to FIGS. 11 and 12. FIG. 13 is a diagram showing an example of a document including a divided image in the present embodiment.

[0078] In the document illustrated in FIG. 11, an image of a slope with a relatively short extension is inserted into the area where the image is to be inserted. Here, it is assumed that the extension of the slope related to the image is 67.1 m. Further, in the document illustrated in FIG. 12, an image of a slope with a relatively long extension is inserted into the area where the image is to be inserted. Here, it is assumed that the extension of the slope related to the image is 212.7 m. When comparing the image inserted in the document of FIG. 11 with the image inserted in the document of FIG. 12, since the extension of the slope in the image of FIG. 12 is longer, when inserted into the same-sized area, the image will be more reduced. Therefore, the image in the document of FIG. 12 has lower visibility than the image in the document of FIG. 11, and it is difficult to grasp the state of the slope, etc.

[0079] Therefore, in the present embodiment, when the extension of the slope is greater than a predetermined threshold value, the image is divided into a plurality of parts, and the divided images are inserted into the document. For example, when the extension of the slope exceeds 100 m, it is assumed that the visibility of the image inserted into the document may decrease, and a process of dividing the image is performed. Therefore, as shown in FIG. 13, with the predetermined threshold value set at 100 m, the image is divided every 100 m from the starting side, and each divided image is inserted into each of a plurality of predetermined areas. In the example shown in FIG. 13, the captured image is generated as a first divided image divided from the starting point to the 100 m position, a second divided image divided from the 100 m position to the 200 m position, and a third divided image divided from the 200 m position to the end point (212.7 m position), and each divided image is inserted into a predetermined area.

[0080] As shown in FIG. 13, by dividing and inserting an image, in addition to a document including the reduced image as shown in FIG. 12, a document including an image with high visibility can be created. Note that the document including the divided image may be created separately from the document of FIG. 12, or may be created in a form in which the divided image is inserted into the document including the image of the entire slope surface as shown in FIG. 12. Also, in the embodiment to be described, a predetermined threshold value for dividing the image is set to 100 m, but any value can be set. For example, when trying to insert an image of an arbitrary shooting range into a document in the prior art, each time, an operator has to go to the site and shoot an image in the desired shooting range, which places a heavy burden on the operator. Or, if the entire slope surface is to be included in a single shooting range, a wide shooting angle is required, and furthermore, the longer the extension of the slope surface, the narrower the range in which the slope surface area is photographed, so the number of pixels in the slope surface area decreases, that is, the image quality of the divided image deteriorates. On the other hand, in the image photographed while scanning in the extension direction of the slope surface as in the present invention, the image can be easily divided at an arbitrary position, and the visibility of the divided image can also be improved.

[0081] In addition, margins may be provided above, below, to the left, and to the right of the frame in the area where the image or the divided image (hereinafter referred to as "image etc.") is inserted. That is, instead of inserting the image etc. according to the size of the frame of the area, the size of the image etc. may be changed so that the image etc. fits within a predetermined range within the frame and inserted into the document. Thereby, the visibility of the image etc. inserted into the document can be improved.

[0082] In addition, from the perspective of improving the visibility of the image, images and the like may be inserted centered in a predetermined area of the document. In this case, if the shooting range of the split image on the far right side among the split images is less than a predetermined threshold value, the split image may be inserted left-aligned in the predetermined area. For example, in the example shown in FIG. 13, the split image in the range from the starting point to the position 100 m and the split image in the range from the position 100 m to the position 200 m are inserted centered in the predetermined area. On the other hand, in the example of FIG. 13, the split image in the range from the position 200 m to the end point has a shooting range of 12.7 m, which is less than the predetermined threshold value of 100 m, and thus is inserted left-aligned in the predetermined area. In this way, by inserting the split image on the far right side left-aligned, the visibility of the split images in the document can be improved compared to the case of inserting them centered.

[0083] Return the explanation to FIG. 8. In step S1006, if the extension of the slope is smaller than a predetermined threshold value (NO), the process proceeds to step S1009. In this case, since it is not necessary to divide the image, the document illustrated in FIG. 11 is created.

[0084] On the other hand, in step S1006, if the extension of the slope is larger than a predetermined threshold value (YES), the process proceeds to step S1007. In step S1007, the split image generation unit 324 performs a process of dividing the captured image into a plurality of images for each predetermined value to generate split images. Here, the predetermined value for dividing the image can be the same value as the predetermined threshold value that serves as the criterion for the branching process in step S1006. For example, in the embodiment to be described, in the case of a slope with an extension exceeding 100 m, the image of the slope is divided in units of 100 m. If the extension of the slope is not an integer multiple of the predetermined threshold value, the split image on the far right side is generated as a split image that has captured a range less than the predetermined value. For example, in the example shown in FIG. 13, the extension of the slope is 212.7 m, and when divided every 100 m, the split image on the far right side is one that has captured the range from 200 m to 212.7 m.

[0085] Next, in step S1008, the document creation unit 326 inserts the divided image generated in step S1007 into a predetermined area of the document (see FIG. 13). In this case, in addition to the divided image, an image of the entire slope surface (for example, the image shown in FIG. 12) may be inserted into the document.

[0086] When the extension of the slope surface in S1006 is smaller than a predetermined threshold value, or after step S1008, in step S1009, the document creation unit 326 inputs various data into each item of the document. Here, the items of the document are, for example, the ID of the slope surface, the extension, the height, the latitudes of the starting point and the ending point, the longitudes, and the like. Also, the extension of the slope surface input into the document can be the value calculated in step S1004. By inputting various data related to the slope surface into the document together with the image in this way, it becomes easier for the viewer of the document to recognize the situation of the slope surface. Note that the process of step S1009 may be performed before or in parallel with the processes of steps S1005 to S1008.

[0087] Thereafter, in step S1010, the document creation unit 326 of the server 120 outputs the created document. The document may be transmitted to the terminal device 130 by communication via, for example, the communication unit 321 and the communication unit 331, and may be displayed on the display 209 of the terminal device 130 by the display control unit 332, or may be printed on a paper medium. Also, the document may be stored in the data storage unit 327 or may be stored in various storage media. By outputting the document in this way, information can be appropriately provided to the administrator of the slope surface or the like.

[0088] Thereafter, in step S1011, the server 120 ends the process of creating the document.

[0089] According to the process shown in FIG. 8, a document including the object can be created. Also, according to the process shown in FIG. 8, according to the extension of the slope surface, the image of the slope surface can be inserted into the document in an appropriate format, and a document with an easy-to-view image of the slope surface can be created.

[0090] Hereinafter, each process shown in FIG. 8 will be described with a sequence diagram showing the communication between the server 120 and the terminal device 130. Note that the sequence diagram shown below is applicable to the information processing system 1 that performs cloud processing and includes the server 120, and is not applicable in a stand-alone form.

[0091] FIG. 14 is a sequence diagram showing the process according to the example of the present embodiment. The process shown in FIG. 14 is started when the terminal device 130 requests evaluation data from the server 120 in step S2001. Here, the evaluation data includes, for example, image data and various sensor data. The server 120 that has received the request in step S2001 transmits the evaluation data to the terminal device 130 in step S2002.

[0092] The terminal device 130 that has received the evaluation data displays a screen (see FIG. 9) for selecting a processing target image in step S2003. Thereafter, the terminal device 130 receives a request to create a document in step S2004. The request to create a document can be received, for example, by pressing the document creation button 902 after selecting the processing target image with the image switching buttons 901L and 902R.

[0093] When receiving a request to create a document in step S2004, the terminal device 130 requests the server 120 to create a document in step S2005. The server 120 that has received the request to create a document performs a process of specifying a slope area in step S2006. The specification of the slope area can be performed, for example, by image analysis. Thereafter, the server 120 performs a process of creating a screen for specifying the slope area in step S2007. Here, the screen for specifying the slope area is, for example, a screen as shown in FIG. 10.

[0094] Thereafter, in step S2008, the server 120 transmits the screen created in step S2007 to the terminal device 130. The terminal device 130 that has received the screen displays the screen on the display 209 in step S2009. Next, in step S2010, the terminal device 130 accepts an operation for specifying a normal plane area. The operation for specifying the normal plane area can be, for example, an operation of moving the start marker Ms and the end marker Me on the screen shown in FIG. 10.

[0095] Next, in step S2011, the terminal device 130 transmits information on the specified normal plane area to the server 120. Based on the information received from the terminal device 130, the server 120 performs a process of determining the normal plane area in step S2012, and performs a process of calculating the extension of the determined normal plane area in step S2013.

[0096] Thereafter, the server 120 performs a process of creating a document in step S2014. The data of the created document is transmitted from the server 120 to the terminal device 130 in step S2015. Also, the data of the created document is stored in the server 120 in step S2016. Note that the processes in steps S2015 and S2016 may be performed in an order reverse to the order shown in FIG. 14, or may be performed in parallel.

[0097] The terminal device 130 that has received the document data in step S2015 performs a process of displaying the received document on the display 209 in step S2017.

[0098] Note that when the specification of the normal plane area is performed only by image analysis by the server 120, that is, when the specification of the normal plane area by the user's operation is not performed, for example, the process can be as shown in the sequence diagram in FIG. 15. FIG. 15 is a sequence diagram showing the process according to another example of the present embodiment.

[0099] The process shown in FIG. 15 is the one obtained by removing from the sequence diagram shown in FIG. 14 the process related to the specification of the normal plane area by the user's operation (the processes of steps S2008 to S2012), and the rest is the same as in FIG. 14. Therefore, the detailed description of each process is omitted.

[0100] By the server 120 and the terminal device 130 performing the processes shown in FIG. 14 or FIG. 15, a document including an image can be created in the form of the cloud.

[0101] As described above, according to the embodiment of the present invention, an information processing apparatus capable of creating a document including an image can be provided.

[0102] Each function of the above-described embodiment of the present invention can be realized by a device-executable program described in C, C++, C#, Java (registered trademark), etc. The program of this embodiment can be stored in a device-readable recording medium such as a hard disk device, CD-ROM, MO, DVD, flexible disk, EEPROM (registered trademark), EPROM, etc. and distributed, and can also be transmitted via a network in a form that other devices can handle.

[0103] Each function of the embodiment described above can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in this specification includes a processor programmed to execute each function by software like a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

[0104] As described above, the present invention has been described with embodiments. However, the present invention is not limited to the above-described embodiments, and as long as the functions and effects of the present invention are achieved within the scope of embodiments that those skilled in the art can conceive, it is included in the scope of the present invention.

Explanation of Signs

[0105] 1, 1’... Information processing system, 10... Imaging system, 110... Imaging device, 120... Server, 130... Terminal device, 140... Network, 150... Information processing device, 201... CPU, 202... RAM, 203... ROM, 204... Storage device, 205... Communication I / F, 206... Camera, 207... GPS device, 208... Distance sensor, 209... Display, 210... Input device, 311... Communication unit, 312... Imaging unit, 313... Position information acquisition unit, 314... Distance measurement unit, 321... Communication unit, 322... Object identification unit, 323... Normal plane extension calculation unit, 324... Divided image generation unit, 325... Latitude and longitude calculation unit, 326... Document creation unit, 327... Data storage unit, 331... Communication unit, 332... Display control unit, 351... Communication unit, 352... Object identification unit, 353... Normal plane extension calculation unit, 354... Divided image generation unit, 355... Latitude and longitude calculation unit, 356... Document creation unit, 357... Data storage unit, 358... Display control unit, 901L, 901R... Image switching buttons, 902... Document creation button 1001... Start position designation button, 1002... End position designation button, 1003... Specific position determination button, 1004... Reduction button, 1005... Enlargement button

Prior Art Documents

Patent Documents

[0106]

Patent Document 1

Claims

1. Identifying means for identifying an object included in an image, Calculating means for calculating the size of the object, Creating means for creating a document of the object, including The creating means cuts out the area of the object in the image and inserts it into a predetermined area of the document, and creates a document in which the size of the object is described in the document. An information processing apparatus.

2. The image is an image taken while moving, and the size of the object is the length in the moving direction when the image is taken. The information processing apparatus according to claim 1.

3. The image is a composite image in which a plurality of images taken while moving are joined together along the moving direction when the plurality of images are taken. The information processing apparatus according to claim 2.

4. When the size of the object is longer than a predetermined size, the creating means generates a divided image obtained by dividing the area of the object in the image for each predetermined size, and inserts the divided image into a predetermined area of the document. The information processing apparatus according to claim 1.

5. The creating means creates a document in which information indicating the positions of the start point and the end point of the object is described in the document. The information processing apparatus according to claim 1.

6. The identifying means identifies an object included in the image by image analysis. The information processing apparatus according to claim 1.

7. A display unit for displaying the image, Display control means for displaying, on the display unit, the image and markers indicating the start point and the end point of the boundary between the area of the object and the area that is not the object in the image, comprising The identifying means is a receiving means for receiving an instruction to move the marker displayed on the display unit, The creating means describes, in the document, the length from the start point to the end point of the object indicated by the marker as the size of the object. The information processing apparatus according to claim 1.

8. The calculating means calculates the size of the object based on the focal length of the lens constituting the camera that took the image, the distance from the camera to the object, the pixel size of the camera, and the number of pixels constituting the object in the image. The information processing apparatus according to claim 1.

9. further including communication means for communicating with a terminal device, The information processing apparatus according to claim 1, which transmits the document to the terminal device via the communication means.

10. transmits the image to the terminal device via the communication means, The information processing apparatus according to claim 9, wherein the specifying means specifies an object included in the image based on an operation performed on the terminal device received via the communication means.

11. An information processing system including a mobile body and a photographing system provided with a camera installed in the mobile body, and an information processing apparatus that processes an image photographed by the photographing system, The photographing system photographs an image while moving with the camera, The information processing apparatus, Specifying means for specifying an object included in the image; Calculating means for calculating the size of the object; Creating means for creating a document of the object; Including, The creating means cuts out the area of the object in the image and inserts it into a predetermined area of the document, and creates a document with the size of the object described in the document.

12. Further including a terminal device capable of communicating with the information processing apparatus, The information processing system according to claim 11, wherein the document is transmitted to the terminal device by communicating with the terminal device.

13. A specifying step of specifying an object included in an image; A calculating step of calculating the size of the object; A creating step of creating a document of the object; Including, In the creating step, an area of the object in the image is cut out and inserted into a predetermined area of the document, and a document with the size of the object described in the document is created.

14. In the information processing apparatus, A specifying step of specifying an object included in an image; A calculating step of calculating the size of the object; A creating step of creating a document of the object; To execute, In the creating step, an area of the object in the image is cut out and inserted into a predetermined area of the document, and a document with the size of the object described in the document is created.

Images