Image processing apparatus, method, and program

Abstract

Provided are an image processing device, method, and program with which it is possible to easily classify and organize images obtained by imaging a structure. An image processing device (10, 50A, 200) comprises: a member distinguishment unit (120, 520, 220) for distinguishing members included in a structure image for distinguishment obtained by imaging the structure; and an identification information addition unit (122, 522, 222) for adding, to the structure image for distinguishment, member identification information indicating the members distinguished by the member distinguishment unit.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, method, and program, and more particularly to an image processing apparatus, method, and program for processing an image obtained by imaging a structure such as a bridge.

Background Art

[0002] Structures such as bridges, roads, tunnels, and dams are developed as the foundation of industry and life and play an important role in supporting people's comfortable lives. Such structures are constructed using, for example, concrete, steel frames, etc. However, since they are used by people over a long period of time, they deteriorate over time. Therefore, it is necessary to regularly inspect such structures to detect damage and deterioration points and perform appropriate maintenance management such as member replacement or repair.

[0003] As a technology for inspecting such structures, Patent Document 1 discloses an inspection system that images a bridge girder or the like of a bridge using a hovering camera configured to automatically fly according to preset flight information. According to Patent Document 1, by automatically flying the hovering camera to image a bridge girder or the like of a bridge, it is possible to perform bridge inspection work without affecting traffic and at low cost.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When inspecting structures such as bridges by imaging them, as in the inspection system described in Patent Document 1, a large number of images are captured each time an inspection is performed. Furthermore, inspection work, including various diagnoses using the images of the structure and the creation of inspection and diagnostic reports, is often performed on a component-by-component basis. Therefore, when performing inspection work, the large number of images obtained by imaging the structure must be classified and organized according to the components that make up the structure.

[0006] One method for classifying and organizing such a large number of images is for inspectors to visually examine the images and classify and organize them according to the components that make up the structure. Another method is to determine the correspondence between the image acquisition location and the design data or 3D model (hereinafter referred to as the 3D model) containing information about the structure of the structure being imaged, and then classify and organize the images using this correspondence.

[0007] However, classifying and organizing a large number of images using the above method is time-consuming. Furthermore, visual inspection by inspectors carries the risk of human error, potentially leading to improper classification and organization of images.

[0008] This invention has been made in view of these circumstances, and aims to provide an image processing device, method, and program that can easily classify and organize images obtained by imaging structures. [Means for solving the problem]

[0009] To solve the above problems, the image processing apparatus according to the first aspect of the present invention comprises a member discrimination unit that discriminates members that are captured in an image of a structure to be discriminated against, and an identification information granting unit that grants member identification information indicating the members discriminated against by the member discrimination unit to the image of the structure to be discriminated against.

[0010] In the first embodiment, the image processing apparatus according to the second aspect of the present invention includes a member discrimination unit which performs image recognition of a structural image to be discriminated and, based on the results of the image recognition, discriminates the members depicted in the structural image to be discriminated.

[0011] In the third aspect of the present invention, the image processing apparatus, in the second aspect, includes a member discrimination unit equipped with a member discrimination AI that has learned how to discriminate members constituting a structure, and uses the member discrimination AI to discriminate members that are captured in the structural image to be discriminated against.

[0012] In the fourth aspect of the present invention, in the second aspect, the member discrimination unit detects a member identifier attached to each member constituting a structure from the structural image to be discriminated, and uses the member identifier to discriminate the member that is captured in the structural image to be discriminated.

[0013] In the fifth aspect of the present invention, in the second aspect, the member discrimination unit performs image recognition on an assigned structure image to which member identification information has already been assigned and on a structure image to be discriminated, and based on the results of the image recognition, it discriminates the members that are depicted in the structure image to be discriminated.

[0014] In the sixth aspect of the present invention, in the second aspect, the member discrimination unit acquires a structural model which includes information indicating the structure of a structure and is associated with member identification information of a structure, and a reference structural image associated with the structural model, performs image recognition on the reference structural image and the structural image to be discriminated, searches for a reference structural image which contains the same member as the member shown in the structural image to be discriminated based on the result of the image recognition, and discriminates the member shown in the structural image to be discriminated based on the correspondence between the reference structural image which contains the same member as the member shown in the structural image to be discriminated and the structural model.

[0015] In the first embodiment of the seventh aspect of the present invention, the image processing apparatus includes a member discrimination unit which acquires reference information for discriminating a member that is depicted in an image of a structure to be discriminated, and which discriminates a member that is depicted in an image of a structure to be discriminated based on the reference information.

[0016] In the eighth aspect of the present invention, in the seventh aspect, the reference information includes a structural model that includes information indicating the structure of a structure, and information regarding the imaging conditions that includes information on the imaging position and imaging direction at the time of imaging of the structural image to be determined, and the member determination unit determines the members that are captured in the structural image to be determined based on the correspondence between the structural model and the information regarding the imaging conditions.

[0017] In the ninth aspect of the present invention, in the seventh aspect, the image processing apparatus includes, as reference information, information relating to the imaging plan of a structure, which includes the imaging order of the structural images to be identified and information indicating the correspondence between the structural images and the members constituting the structure, and the member identification unit identifies the members that are depicted in the structural images to be identified based on the correspondence between the imaging plan information and the structural images to be identified.

[0018] In the ninth embodiment, the image processing apparatus according to the tenth aspect of the present invention includes, in the image processing apparatus, information relating to the imaging plan includes information indicating the imaging position and imaging direction when imaging a structure, and the image processing apparatus comprises an imaging unit for imaging a structure, a positioning unit for measuring the imaging position and imaging direction of the imaging unit, and a control unit for controlling the imaging position and imaging direction of the imaging unit so that they match the information indicating the imaging position and imaging direction included in the information relating to the imaging plan.

[0019] In the eleventh aspect of the present invention, in any of the first to tenth aspects, the identification information assigning unit assigns member identification information relating to multiple members to the structural image to be identified when multiple members are depicted in the structural image to be identified.

[0020] In the twelfth aspect of the present invention, the image processing apparatus according to any one of the first to eleventh aspects includes a damage information adding unit that detects damage from a structure image and adds information related to the damage to the structure image.

[0021] In the thirteenth aspect of the present invention, the image processing apparatus according to any one of the first to twelfth aspects includes an added information editing unit for editing the added information added to the structure image.

[0022] In the fourteenth aspect of the present invention, the image processing apparatus according to any one of the first to thirteenth aspects includes a structure image search unit that searches for a structure image based on the added information added to the structure image.

[0023] The image processing method according to the fifteenth aspect of the present invention includes a step of discriminating a member shown in a structure image of a discrimination target that has imaged a structure, and a step of adding member identification information indicating the member discriminated as a member shown in the structure image of the discrimination target to the structure image of the discrimination target.

[0024] The image processing program according to the sixteenth aspect of the present invention causes a computer to realize a member discrimination function for discriminating a member shown in a structure image of a discrimination target that has imaged a structure, and an identification information adding function for adding member identification information indicating the member discriminated by the member discrimination function to the structure image of the discrimination target.

[0025] Note that each of the above aspects can also be realized as an image processing apparatus for a computer processor to realize the functions of each of the above units. That is, it is also possible to realize that the processor of the image processing apparatus discriminates a member shown in a structure image of a discrimination target that has imaged a structure, and adds member identification information indicating the discriminated member to the structure image of the discrimination target.

Advantages of the Invention

[0026] According to the present invention, by discriminating a member shown in a structure image and adding member identification information to the structure image, it becomes possible to easily classify and organize the structure images.

Brief Description of the Drawings

[0027] [Figure 1] FIG. 1 is a block diagram showing an image processing apparatus according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a diagram for explaining an image processing function according to a first embodiment of the present invention. [Figure 3] FIG. 3 is a flowchart showing an image processing method according to a first embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart showing a member discrimination step (step S12 in FIG. 3) according to a first embodiment of the present invention. [Figure 5] FIG. 5 is a diagram for explaining an image processing function according to a second embodiment of the present invention. [Figure 6] FIG. 6 is a flowchart showing a member discrimination step (step S12 in FIG. 3) according to a second embodiment of the present invention. [Figure 7] FIG. 7 is a diagram for explaining an image processing function according to a third embodiment of the present invention. [Figure 8] FIG. 8 is a perspective view showing a specific example of a structure model. [Figure 9] FIG. 9 is an enlarged plan view of region IX in FIG. 8 as viewed from below. [Figure 10] FIG. 10 is a perspective view showing an enlarged region X in FIG. 8. [Figure 11] FIG. 11 is a diagram for explaining member discrimination using a structure image associated with the structure in FIG. 10. [Figure 12] FIG. 12 is a flowchart showing a member discrimination step (step S12 in FIG. 3) according to a third embodiment of the present invention. [Figure 13] FIG. 13 is a diagram for explaining an image processing function according to a fourth embodiment of the present invention. [Figure 14] FIG. 14 is a diagram for explaining member discrimination using information on a structure model and imaging conditions. [Figure 15]Figure 15 is a flowchart showing the component identification process (step S12 in Figure 3) according to the fourth embodiment of the present invention. [Figure 16] Figure 16 is a diagram illustrating the image processing function according to the fifth embodiment of the present invention. [Figure 17] Figure 17 is a diagram illustrating member identification using information about the structural model and imaging plan. [Figure 18] Figure 18 is a flowchart showing the component identification process (step S12 in Figure 3) according to the fifth embodiment of the present invention. [Figure 19] Figure 19 is a block diagram showing an image processing function according to an additional embodiment. [Figure 20] Figure 20 is a block diagram showing an imaging device equipped with an image processing function according to Modification Example 1. [Figure 21] Figure 21 is a block diagram showing a cloud server equipped with image processing capabilities according to Modification Example 2. [Modes for carrying out the invention]

[0028] Preferred embodiments of the image processing apparatus, method, and program according to the present invention will be described below with reference to the attached drawings.

[0029] [First Embodiment] (Image processing device) Figure 1 is a block diagram showing an image processing apparatus according to the first embodiment of the present invention.

[0030] The image processing device 10 according to this embodiment is a device for acquiring structural images D1, which are images of each part of the structure OBJ to be inspected, from the imaging device 50, and for classifying and organizing them.

[0031] The imaging device 50 is equipped with a camera (for example, a CCD (Charge Coupled Device) camera or an infrared camera) for capturing images of various parts of the OBJ of the structure to be inspected. The imaging device 50 may also be configured to capture images while moving around the OBJ of the structure to be inspected, for example, by mounting a camera on a mobile body such as an unmanned aerial vehicle like a multicopter or drone, a vehicle, or a robot.

[0032] As shown in Figure 1, the image processing apparatus 10 according to this embodiment includes a control unit 12, an input unit 14, a display unit 16, a storage unit 18, and a communication interface (communication I / F: interface) 20. The image processing apparatus 10 may be, for example, a general-purpose computer such as a personal computer or workstation, or a tablet terminal.

[0033] The control unit 12 includes a processor (e.g., a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit)) that controls the operation of each part of the image processing device 10. The control unit 12 is capable of sending and receiving control signals and data to and from each part of the image processing device 10 via a bus. The control unit 12 receives instruction input from the operator via the input unit 14 and transmits control signals corresponding to this instruction input to each part of the image processing device 10 via the bus to control the operation of each part.

[0034] The control unit 12 includes RAM (Random Access Memory), which is used as a work area for various calculations, and VRAM (Video Random Access Memory), which is used as an area for temporarily storing image data output to the display unit 16.

[0035] The input unit 14 is an input device that receives instructions from the operator and includes a keyboard for character input, a pointing device (e.g., mouse, trackball, etc.) for operating the GUI (Graphical User Interface) such as pointers and icons displayed on the display unit 16. Alternatively, the input unit 14 may include a touch panel on the surface of the display unit 16 in addition to the keyboard and pointing device.

[0036] The display unit 16 is a device for displaying images. For example, a liquid crystal monitor can be used as the display unit 16.

[0037] The storage device 18 stores various data, including control programs and image processing programs P1 for various calculations, and structural images D1 (e.g., visible light images or infrared images) of the structure OBJ to be inspected. The storage device 18 can be, for example, a device including a magnetic disk such as an HDD (Hard Disk Drive), or a device including flash memory such as an eMMC (embedded Multi Media Card) or SSD (Solid State Drive).

[0038] The communication interface 20 is a means for communicating with external devices, including the imaging device 50, via a network. For sending and receiving data between the image processing device 10 and the external devices, wired communication or wireless communication (e.g., LAN (Local Area Network), WAN (Wide Area Network), Internet connection, etc.) can be used.

[0039] The image processing device 10 can receive input of the structure image D1 from the imaging device 50 via the communication I / F 20. Note that the method of inputting the structure image D1 to the image processing device 10 is not limited to communication via a network. For example, a USB (Universal Serial Bus) cable, Bluetooth®, infrared communication, etc., may be used, or the structure image D1 may be stored in a detachable and readable recording medium (e.g., a USB memory stick) attached to the image processing device 10, and the image processing device 10 may receive input of the structure image D1 via this recording medium.

[0040] (Image processing function) Next, the image processing functions (component discrimination function and identification information assignment function) of the image processing device 10 will be explained with reference to Figure 2. Figure 2 is a diagram illustrating the image processing functions.

[0041] The processor of the control unit 12 can realize the functions of the component discrimination unit 120 and the identification information assignment unit 122 by reading and executing the image processing program P1 from the storage 18.

[0042] As shown in Figure 2, the member discrimination unit 120 is equipped with a member discrimination AI 120A (Artificial Intelligence). The member discrimination unit 120 uses the member discrimination AI 120A to perform image recognition or image analysis of the structure image D1 to identify the members that make up the structure OBJ shown in the structure image D1.

[0043] Here, the component discrimination AI 120A is created using supervised learning (e.g., a classifier) ​​that learns the relationship between input and output data by using training data that takes images of a part of a structural object (OBJ) such as a bridge as input and outputs the name of that component. Note that the learning algorithm of the component discrimination AI 120A is not limited to supervised learning, but may also be unsupervised learning.

[0044] Furthermore, the images used to train the component discrimination AI 120A do not need to be images of the same structural OBJ as the object being inspected. For example, images of similar or identical structural OBJs, or images created using design data (e.g., a 3D model) of the structural OBJ, can be used to train the component discrimination AI 120A.

[0045] The identification information assignment unit 122 obtains the identification result of the members shown in the structural image D1 from the member identification unit 120 and assigns member identification information D2 to the structural image D1.

[0046] The component identification information D2 includes, for example, the type of component that constitutes the structural OBJ (hereinafter referred to as "component type") and an identifier for identifying the component in the structural OBJ (hereinafter referred to as "component ID (Identification)").

[0047] Member classification refers to the classification of members that make up a structural object-oriented building (OBJ) based on their shape, function, material, or dimensions. For example, in the case of a bridge, member classifications include main girders, cross girders, piers, and deck slabs.

[0048] A member ID is an identifier used to identify which part of a structural object-oriented building (OBJ) a member is used for. A member ID may be defined for one of several main girders used at position A, for example, as main girder A-1, or it may be defined by specific position coordinates (for example, the position coordinates of the design centroid or the position coordinates of the end).

[0049] A structural object-oriented document (OBJ) may be composed of multiple identical members of the same member type. Therefore, by using member identification information D2, which combines the member type and member ID, it becomes possible to identify the type and arrangement of members shown in the structural image D1. However, if the member ID includes information on both the member type and arrangement, as exemplified by the main girder A-1 above, then only the member ID may be used as member identification information D2.

[0050] Component identification information D2 can be stored within the Exif (Exchangeable Image File Format) if the structural image D1 is an Exif file. Specifically, component identification information D2 can be stored in association with tags related to operator information, such as MakerNote (a tag used by manufacturers to add individual information for their own use) or UserComment (a tag within the Exif information). Alternatively, component identification information D2 can be stored within a custom application marker segment (APPn) added to the Exif file.

[0051] (Image processing method) Next, the image processing method according to this embodiment will be described with reference to Figures 3 and 4. Figure 3 is a flowchart of the image processing method according to the first embodiment of the present invention.

[0052] First, the control unit 12 reads the structural image D1 acquired from the imaging device 50 from the storage 18 (step S10).

[0053] Next, the component discrimination unit 120 performs image recognition on the structural image D1 to be discriminated, which was read in step S10, and identifies the components that make up the structural OBJ shown in the structural image D1 (step S12).

[0054] In step S12, first, as shown in Figure 4, the member discrimination unit 120 uses the member discrimination AI 120A to perform image recognition on the structure image D1 and discriminates the members that make up the structure OBJ shown in the structure image D1 (step S20). Then, the member discrimination unit 120 outputs the member discrimination result to the identification information assignment unit 122 (step S22).

[0055] Next, the identification information assignment unit 122 obtains the identification result of the members shown in the structural image D1 from the member identification unit 120 and assigns member identification information to the structural image D1 (step S14).

[0056] Then, steps S12 to S14 are repeated until all structural images D1 have been identified (step S16).

[0057] According to this embodiment, by performing image recognition of the structure image D1 using the member discrimination AI 120A, member identification information D2 can be added to the structure image D1. This makes it possible to easily classify and organize the structure image D1.

[0058] In this embodiment, the member identification information D2 is stored in the data (Exif information) of the structure image D1, but the present invention is not limited to this. For example, instead of storing the member identification information in the data of the structure image D1, it may be stored in the application for viewing the structure image D1, and the member identification information D2 may be displayed when viewing the structure image D1 using the viewing application.

[0059] Furthermore, the structural image D1 and the member identification information D2 do not necessarily have to be in a one-to-one correspondence. That is, if multiple members are detected from the structural image D1, member identification information D2 for multiple members may be stored in the structural image D1.

[0060] The component identification information D2 may be divided into main components and non-main components. Here, the criteria for determining whether a component is a main component can be, for example, its position in the structural image D1 or the area it occupies in the structural image D1.

[0061] Specifically, the main members may be those positioned closest to the center of the field of view in the structural image D1, or those that occupy the largest area in the structural image D1. Furthermore, members located at the edges of the field of view in the structural image D1, or those occupying a small area in the structural image D1 (for example, members occupying 1% to 5% or less of the area of ​​the structural image D1), may be considered non-main members or may not be included in the member identification information D2.

[0062] By identifying the main components in component identification information D2, the classification and organization of structural images D1 can be facilitated.

[0063] (Modified version of the first embodiment) In this embodiment, image recognition of the structure image D1 is performed using the member discrimination AI 120A, but the present invention is not limited thereto. For example, it is also possible to distinguish members using a member identifier (member identification mark; for example, a one-dimensional code, a two-dimensional code, or a QR (Quick Response) code (registered trademark)) defined for each member of the structure OBJ.

[0064] Specifically, a member identification mark, defined for each member of the structural OBJ, is attached to each member of the structural OBJ.

[0065] Furthermore, storage 18 stores a lookup table that shows the correspondence between component identification marks and components.

[0066] Furthermore, by using image recognition by the component identification unit 120 to detect component identification marks from the structural image D1, it is possible to identify the components.

[0067] [Second Embodiment] Next, a second embodiment of the present invention will be described. In the following description, the same configuration as in the above embodiment will be omitted from the description, and only the part of the image processing function and image processing method related to member discrimination will be described.

[0068] (Image processing function) Figure 5 is a diagram illustrating the image processing function according to a second embodiment of the present invention.

[0069] The member identification unit 120 in this embodiment can refer to a pre-assigned structure image D3, to which member identification information D2 has already been assigned, as reference information. The member identification unit 120 then identifies the members constituting the structure OBJ shown in the structure image D1 by comparing (matching) the structure image D1 to be identified with the pre-assigned structure image D3.

[0070] The assigned structure image D3 may be stored in the storage 18 of the image processing device 10, or it may be stored in external storage accessible by the image processing device 10 (for example, cloud storage).

[0071] As shown in Figure 5, the member discrimination unit 120 is equipped with an image search engine 120B. The member discrimination unit 120 uses the image search engine 120B to perform image recognition on the structural image D1 to be discriminated against and searches for images containing the same member from the assigned structural images D3.

[0072] Specifically, the image search engine 120B extracts feature points from the structure image D1 to be identified, matches them with the feature points extracted from each image of the assigned structure image D3, and searches for images of the same member within the structure image D3. Then, the member identification unit 120 reads the member identification information D2 of the image found within the assigned structure image D3, and identifies the member that constitutes the structure OBJ shown in the structure image D1.

[0073] Furthermore, it is also possible to use a similar image search engine that employs machine learning or deep learning as the image search engine 120B.

[0074] The identification information assignment unit 122 obtains the identification result of the members shown in the structural image D1 from the member identification unit 120 and assigns member identification information D2 to the structural image D1.

[0075] (Image processing method) Figure 6 is a flowchart showing the component identification process (step S12 in Figure 3) according to the second embodiment of the present invention.

[0076] First, the control unit 12 reads the assigned structure image D3 from the storage 18 or external storage (step S30). Here, the assigned structure image D3 may be an image of the structure OBJ to be inspected, or it may be an image of a structure of the same type or similar to the structure OBJ to be inspected.

[0077] Next, the member discrimination unit 120 uses the image search engine 120B to perform image recognition on the structure image D1 to be discriminated and compares the structure image D1 with the assigned structure image D3 (step S32). Then, the member discrimination unit 120 searches the assigned structure image D3 for images that show the same member as the structure image D1 (step S34).

[0078] Next, the member identification unit 120 reads the member identification information D2 of the image found from the assigned structural image D3, thereby identifying the members that make up the structural OBJ shown in the structural image D1 (step S36).

[0079] The component identification unit 120 then outputs the component identification result to the identification information assignment unit 122 (step S38). This allows component identification information to be assigned to the structure image D1 (step S14 in Figure 3).

[0080] According to this embodiment, member identification information D2 can be assigned to a structure image D1 by referring to a structure image D3 to which member identification information D2 has already been assigned. This makes it possible to easily classify and organize the structure image D1.

[0081] [Third Embodiment] Next, a third embodiment of the present invention will be described. In the following description, the same configuration as in the above embodiment will be omitted from the description, and only the part of the image processing function and image processing method related to member discrimination will be described.

[0082] (Image processing function) Figure 7 is a diagram illustrating the image processing function according to a third embodiment of the present invention.

[0083] The member discrimination unit 120 according to this embodiment includes an image search engine 120C. The member discrimination unit 120 can refer to a structural model D41 to which member identification information D2 is assigned, and a reference structural image D42 associated with the structural model D41, as reference information. The member discrimination unit 120 uses the image search engine 120C to refer to the structural image D1 to be discriminated against, the structural model D41, and the reference structural image D42, thereby discriminating the members that constitute the structural OBJ shown in the structural image D1.

[0084] Here, the structural model D41 is data containing information about the shape and structure of the structure OBJ to be inspected, for example, a 3D model containing information about the 3D shape of the structure OBJ to be inspected. The structural model D41 may be design data of the structure OBJ to be inspected, or it may be measured data obtained in advance. For example, CAD (Computer-Aided Design) data or 3D CAD data can be used as the structural model D41. Furthermore, when using measured data as the structural model D41, it is also possible to use point cloud data obtained by reconstructing the 3D shape from multi-view images of the structure OBJ using SfM (Structure from Motion) technology.

[0085] The structural model D41 is assigned member identification information D2 for each member that makes up the structural OBJ. This member identification information D2 is associated with the coordinates in the CAD data or the point cloud data and stored in the structural model D41.

[0086] The reference structure image D42 is an image obtained by capturing the structure's OBJ, and is stored in association with the position coordinates of the structure's OBJ in the structure model D41.

[0087] The structural model D41 and the reference structural image D42 may be stored in the storage 18 of the image processing device 10, or they may be stored in external storage accessible by the image processing device 10 (for example, cloud storage).

[0088] Here, specific examples of the structural model D41 and the reference structural image D42 will be explained with reference to Figures 8 to 11.

[0089] Figure 8 is a perspective view showing a specific example of a structural model, and Figure 9 is an enlarged plan view of area IX in Figure 8, viewed from below. Figures 8 and 9 illustrate a 3D model D41(OBJ1) of a bridge OBJ1 as a specific example of a structural OBJ.

[0090] As shown in Figures 8 and 9, in the 3D model D41(OBJ1) of the bridge OBJ1, member identification information D2 is assigned to each member that makes up the bridge OBJ1. The reference structure image D42 is associated with each part of the 3D model D41(OBJ1).

[0091] In Figures 8 and 9, the component identification information D2 shows the component type and component ID. However, if the component ID includes information about the component type, the component type can be omitted.

[0092] Figure 10 is a perspective view showing an enlarged view of area X (bridge pier OBJ2) in Figure 8, and Figure 11 is a diagram illustrating member identification using reference structural image D42 associated with structural OBJ2 in Figure 10.

[0093] Figure 10 shows an overall image of pier OBJ2, which is part of bridge OBJ1, and Figure 11 shows structural images corresponding to regions XI-1 to XI-6 in Figure 10. Note that the imaging direction when the reference structural image D42 shown in Figure 11 was captured may differ from that of the overall image shown in Figure 10.

[0094] As shown in Figure 11, the member discrimination unit 120 uses the image search engine 120C to perform image recognition on the structural image D1 to be discriminated and searches for images containing the same member from the reference structural image D42.

[0095] Specifically, the image search engine 120C extracts feature points from the structure image D1 to be identified and matches them with feature points extracted from each image of the reference structure image D42 to search for images of the same member within the reference structure image D42. Then, the member identification unit 120 reads the member identification information D2 of the structure model D41 associated with the image found in the reference structure image D42 to identify the member that makes up the structure OBJ shown in the structure image D1.

[0096] Furthermore, it is also possible to use a similar image search engine that employs machine learning or deep learning as the image search engine 120C.

[0097] The identification information assignment unit 122 obtains the identification result of the members shown in the structural image D1 from the member identification unit 120 and assigns member identification information D2 to the structural image D1.

[0098] (Image processing method) Figure 12 is a flowchart showing the component identification process (step S12 in Figure 3) according to the third embodiment of the present invention.

[0099] First, the control unit 12 reads the structure model D41 and the reference structure image D42 from the storage 18 or external storage (step S40). Here, the structure model D41 and the reference structure image D42 may be the structure model D41 and reference structure image D42 created for the structure OBJ to be inspected, or they may be the structure model D41 and reference structure image D42 created for a structure of the same or similar type as the structure OBJ to be inspected.

[0100] Next, the member discrimination unit 120 uses the image search engine 120C to perform image recognition on the structure image D1 to be discriminated and compares the structure image D1 with the reference structure image D42 associated with the structure model D41 (step S42). Then, the member discrimination unit 120 searches the reference structure image D42 for images that show the same member as the structure image D1 (step S44).

[0101] Next, the member identification unit 120 reads the member identification information D2 of the structural model D41 associated with the image searched from the reference structural image D42, thereby identifying the members that make up the structural OBJ shown in the structural image D1 (step S46).

[0102] The component identification unit 120 then outputs the component identification result to the identification information assignment unit 122 (step S48). This allows component identification information to be assigned to the structure image D1 (step S14 in Figure 3).

[0103] According to this embodiment, member identification information D2 can be assigned to a structural image D1 by referring to a structural model D41 and a reference structural image D42 to which member identification information D2 has already been assigned. This makes it possible to easily classify and organize the structural image D1.

[0104] [Fourth Embodiment] Next, a fourth embodiment of the present invention will be described. In the following description, the same configuration as in the above embodiments will be omitted from the description, and only the part of the image processing function and image processing method related to component discrimination will be described.

[0105] (Image processing function) Figure 13 is a diagram illustrating the image processing function according to the fourth embodiment of the present invention.

[0106] The member discrimination unit 120 according to this embodiment includes an imaging target calculation unit 120D. The member discrimination unit 120 can refer to a structural model D5 and information D6 regarding the imaging status of the structural image D1 as reference information.

[0107] The member discrimination unit 120 according to this embodiment discriminates members that are visible in the structural image D1 based on the correspondence between the imaging conditions information D6, which includes the imaging position and imaging direction of the structural image D1, and the structural model D5.

[0108] Here, the structural model D5 is data containing information about the shape and structure of the structure OBJ to be inspected, for example, a 3D model containing information about the 3D shape of the structure OBJ to be inspected. The structural model D5 may be design data of the structure OBJ to be inspected, or it may be measured data obtained in advance. For example, CAD data or 3D CAD data can be used as the structural model D5. Also, when using measured data as the structural model D5, it is possible to use point cloud data obtained by reconstructing the 3D shape from multi-view images of the structure OBJ using SfM technology, for example.

[0109] The structural model D5 is assigned member identification information D2 for each member that makes up the structural OBJ. This member identification information D2 is associated with the coordinates in the CAD data or the point cloud data and stored in the structural model D5.

[0110] The imaging condition information D6 is information regarding the imaging conditions when the structural image D1 is captured. The imaging condition information D6 includes, for example, information indicating the position of the imaging device 50 at the time of imaging (hereinafter referred to as the imaging position) and information indicating the orientation of the imaging device 50 at the time of imaging (hereinafter referred to as the imaging direction). The imaging condition information D6 is measured, for example, by a positioning unit including a GPS (Global Positioning System) device, an inertial sensor, and an altitude sensor provided on the imaging device 50.

[0111] The imaging information D6 may be included in the structural image D1 as supplementary information (e.g., Exif information), or it may be obtained from the imaging device 50 as a separate file associated with the structural image D1.

[0112] Figure 14 is a diagram illustrating member identification using the structural model D5 and information D6 regarding the imaging conditions. In Figure 14, the structural model D5 is an example using the 3D model D5 (OBJ1) of bridge OBJ1 (see Figure 8).

[0113] As shown in Figure 14, the imaging information D6 includes information on the imaging position and imaging direction.

[0114] The imaging location is represented, for example, in a 3D xyz orthogonal coordinate system. The imaging location may be represented by coordinates determined by latitude and longitude based on GPS (e.g., global coordinates), or by local coordinates based on a predetermined location on the bridge OBJ1.

[0115] The imaging direction is represented, for example, by an abc 3D Cartesian coordinate system set with the imaging position as the origin and the camera provided on the imaging device 50 as the reference.

[0116] Figure 14 shows the 3D model D5(OBJ1) of bridge OBJ1, illustrating the points indicating the imaging positions, arrows indicating the imaging direction, and rectangles indicating the imaging range for each of the captured images (structure images) IMG1 to IMG4. The imaging range will be discussed later.

[0117] (Example of part identification procedure 1) Specifically, for example, first, we define an imaging direction vector that extends along the imaging direction, starting from the imaging position included in the imaging condition information D6.

[0118] Next, the component discrimination unit 120 uses the imaging target calculation unit 120D to calculate the position where the imaging direction vector first makes contact with the structural model D5 when extended.

[0119] The component identification unit 120 then identifies the component corresponding to the position where the imaging direction vector first contacts the structural model D5 as a component that is captured in the structural image D1.

[0120] (Second example of component identification) Alternatively, the component discrimination unit 120 uses the imaging target calculation unit 120D to calculate the imaging distance between the imaging device 50 and the structure OBJ1 at the time of imaging, based on the correspondence between the imaging position and imaging direction information contained in the imaging status information D6 and the structure model D5. Specifically, the imaging distance may be the distance between the imaging device 50 at the time of imaging and the focus position on the structure OBJ1.

[0121] Next, the component discrimination unit 120 uses the imaging target calculation unit 120D to calculate the imaging range, which is the area captured in the structural image D1, based on the imaging distance, the focal position of the camera lens of the imaging device 50 at the time of imaging, and the size of the image sensor of the camera of the imaging device 50.

[0122] Specifically, if the imaging distance is D, the lens focal length is F, and the sensor size is ((horizontal direction), (vertical direction)) = (Sx, Sy), then the imaging range is expressed by the following formula.

[0123] Image range (horizontal direction) = D × Sx / F Image area (vertical direction) = D × Sy / F Next, the component identification unit 120 identifies components that are assumed to be captured in the imaging range based on the correspondence between the imaging range and the structural model D5 as components captured in the structural image D1.

[0124] Furthermore, if it is assumed that multiple components are present within the imaging range, the component with the largest area within the imaging range will be identified as the component present in structural image D1. Alternatively, multiple components assumed to be present within the imaging range may be identified as components present in structural image D1.

[0125] (Image processing method) Figure 15 is a flowchart showing the component identification process (step S12 in Figure 3) according to the fourth embodiment of the present invention.

[0126] First, the control unit 12 reads the structure model D5 from the storage 18 or external storage. The control unit 12 also reads the imaging status information D6 associated with the structure image D1 (step S50). Here, the imaging status information D6 may be stored as supplementary information to the structure image D1, or it may be stored in a separate file associated with the structure image D1.

[0127] Next, the component identification unit 120 identifies the components shown in the structural image D1 based on the correspondence between the imaging conditions information D6, which includes the imaging position and imaging direction of the structural image D1, and the structural model D5 (step S52).

[0128] The component identification unit 120 then outputs the component identification result to the identification information assignment unit 122 (step S54). This allows component identification information to be assigned to the structure image D1 (step S14 in Figure 3).

[0129] According to this embodiment, by referring to the structural model D5 and the imaging status information D6, member identification information D2 can be added to the structural image D1. This makes it possible to easily classify and organize the structural image D1.

[0130] [Fifth Embodiment] Next, a fifth embodiment of the present invention will be described. In the following description, the same configuration as in the above embodiments will be omitted from the explanation, and only the part of the image processing function and image processing method related to member discrimination will be described.

[0131] (Image processing function) Figure 16 is a diagram illustrating the image processing function according to the fifth embodiment of the present invention.

[0132] The member discrimination unit 120 according to this embodiment includes an imaging target calculation unit 120E. The member discrimination unit 120 can reference a structural model D5 and information D7 related to the imaging plan of the structural image D1 as reference information.

[0133] In this embodiment, the member discrimination unit 120 discriminates the members shown in the structural image D1 based on the correspondence between the information D7 regarding the imaging plan of the structural image D1 and the structural model D5.

[0134] Here, the imaging plan information D7 is information regarding the imaging plan for the structural image D1. The imaging plan information D7 includes, for example, information showing the correspondence between information regarding the imaging sequence of the structural OBJ and information for identifying the members to be imaged (e.g., member ID).

[0135] The imaging plan information D7 may include information indicating the imaging position of the imaging device 50 during imaging and information indicating the imaging direction of the imaging device 50 during imaging. The imaging position and imaging direction are the same as in the fourth embodiment, so their explanation is omitted.

[0136] The imaging plan information D7 may be included in the structural image D1 as supplementary information (e.g., Exif information), or it may be obtained from the imaging device 50, etc., as a separate file associated with the structural image D1.

[0137] Figure 17 is a diagram illustrating member identification using the structural model D5 and imaging plan information D7.

[0138] As shown in Figure 17, the imaging plan information D7 includes information on the imaging sequence of the structure OBJ, information indicating the imaging position of the imaging device 50 during imaging, information indicating the imaging direction of the imaging device 50 during imaging, and member IDs.

[0139] Meanwhile, the storage 18 of the image processing device 10 stores the structure images D1, and the structure images D1 can be sorted according to the imaging order, i.e., the date and time of acquisition.

[0140] As shown in Figure 17, based on the imaging order of the imaging plan information D7 and the imaging order of the structure images D1, the member identification unit 120 uses the imaging target calculation unit 120E to associate the imaging order, the structure images D1 sorted based on the imaging order, and the member IDs. This makes it possible to identify the members that are captured in each structure image D1.

[0141] (Image processing method) Figure 18 is a flowchart showing the component identification process (step S12 in Figure 3) according to the fifth embodiment of the present invention.

[0142] First, the control unit 12 reads the structure model D5 from the storage 18 or external storage. The control unit 12 also reads the imaging plan information D7 (step S60). Here, the imaging plan information D7 may be stored as supplementary information to the structure image D1, or it may be stored in a separate file associated with the structure image D1.

[0143] Next, the component identification unit 120 identifies the components shown in the structural image D1 based on the correspondence between the information D7 regarding the imaging plan of the structural image D1 and the structural model D5 (step S62).

[0144] The component identification unit 120 then outputs the component identification result to the identification information assignment unit 122 (step S64). This allows component identification information to be assigned to the structure image D1 (step S14 in Figure 3).

[0145] According to this embodiment, member identification information D2 can be added to the structural image D1 by referring to information regarding the structural model D5 and the imaging plan. This makes it possible to easily classify and organize the structural image D1.

[0146] In this embodiment, the image processing device 10 may add identification information indicating the imaging order to the structural image D1 file after imaging, or the imaging device 50 may add the identification information immediately after imaging.

[0147] [Additional Implementations] The following forms can be added to each of the above embodiments.

[0148] Figure 19 is a block diagram showing an image processing function according to an additional embodiment.

[0149] In the example shown in Figure 19, the control unit 12 is capable of realizing the functions of the damage information assignment unit 124, the assignment information editing unit 126, and the structure image search unit 128.

[0150] The damage information assignment unit 124 detects damage from the structure image D1 and assigns damage information to the structure image D1. Here, the damage information includes, for example, information such as the location of the damage (information indicating where the damage is located in the structure image D1 (for example, coordinates in the structure image D1)), type, size, or extent (progression).

[0151] The damage information assignment unit 124 can, for example, be a classifier created by supervised learning, which learns the relationship between input and output data using training data that takes images of damage as input and damage information as output. The learning algorithm of the damage information assignment unit 124 is not limited to supervised learning; it may also be unsupervised learning.

[0152] Damage information may be included in structural image D1 as supplementary information (e.g., Exif information), or it may be in a separate file associated with structural image D1.

[0153] The information editing unit 126 is a means for editing information such as member identification information D2 and damage information to be added to the structural image D1. The information editing unit 126 allows for editing of the information added to the structural image D1 either before or after the fact.

[0154] In other words, when the information to be added is to be edited in advance, the information to be added editing unit 126 displays the structure image D1 and the information to be added on the display unit 16 before adding the information to the structure image D1. Then, the information to be added editing unit 126 adds the information to the structure image D1 according to the approval of the addition or the instruction input for editing the information to be added from the input unit 14.

[0155] Furthermore, if the assigned information needs to be edited afterward, the assigned information editing unit 126 displays the structure image D1 with the assigned information already applied and the assigned information on the display unit 16. Then, the assigned information editing unit 126 edits the assigned information already applied to the structure image D1 according to the instructions for editing the assigned information input from the input unit 14.

[0156] This allows the user to edit the information to be added to the structural image D1 via the input unit 14.

[0157] The structural image search unit 128 receives a search key input from the input unit 14, searches for a structural image D1 corresponding to the search key, and displays it on the display unit 16. Here, as the search key, member identification information D2 or damage information can be used as the associated information.

[0158] Each of the above embodiments of the image processing apparatus 10 can be further configured to include at least one additional function of an additional embodiment, such as a damage information assignment unit 124, an assignment information editing unit 126, and a structure image search unit 128.

[0159] [Example 1] In the above embodiments, examples were described in which the image processing device 10 processes images acquired from the imaging device 50, but the present invention is not limited thereto. For example, an imaging device for capturing images of structures can also be configured to include the image processing device and image processing functions according to the above embodiments.

[0160] (Imaging device) Figure 20 is a block diagram showing an imaging device (image processing device) equipped with an image processing function according to Modification Example 1.

[0161] The imaging device 50A according to Modification 1 is, for example, an unmanned aerial vehicle such as a multirotor or drone equipped with a camera 54. The imaging device 50A is capable of wireless communication with the controller 70 and performs flight and imaging in accordance with control signals from the controller 70.

[0162] Furthermore, the imaging device 50A according to Modification 1 includes the image processing functions according to each of the above embodiments and is an example of an image processing device according to the present invention.

[0163] As shown in Figure 20, the imaging device 50A according to this embodiment includes a control unit 52, a camera 54, a drive unit 56, a positioning unit 58, a battery 60, a storage 62, and a communication interface (communication I / F) 64.

[0164] The control unit 52 includes a processor (e.g., a CPU or GPU) that controls the operation of each part of the imaging device 50A, and RAM used as a work area for various calculations.

[0165] The processor of the control unit 52 reads and executes the image processing program P1 from the storage 62, thereby enabling the functions of the component discrimination unit 520 and the identification information assignment unit 522. Note that the component discrimination unit 520 and the identification information assignment unit 522 are the same as the component discrimination unit 120 and the identification information assignment unit 122 in the above embodiments, and therefore their descriptions are omitted.

[0166] The camera (imaging unit) 54 is for capturing an image D1 of the structure OBJ of the structure to be inspected, and includes, for example, a CCD camera or an infrared camera.

[0167] The drive unit 56 includes a motor for rotating a propulsion device such as a propeller 66 attached to the imaging device 50A, and an ESC (Electric Speed ​​Controller) for controlling the rotation speed of the motor. By using this motor to rotate the propeller 66, the drive unit 56 can obtain the lift and thrust necessary to fly the imaging device 50A.

[0168] The positioning unit 58 is a device that acquires position information indicating the position of the imaging device 50A and attitude information indicating its attitude. The positioning unit 58 includes, for example, a GPS device, an inertial sensor, and an altitude sensor. The GPS device performs three-dimensional positioning of the imaging device 50A based on signals transmitted from GPS satellites and acquires position information (e.g., latitude and longitude). The inertial sensor includes, for example, a three-axis accelerometer or a three-axis gyroscope and acquires information indicating the flight status of the imaging device 50A (e.g., speed information, acceleration information, and attitude information). The altitude sensor includes, for example, a barometric, GPS, laser, or radar type altimeter. This GPS position information, flight status information, and altitude information are stored in the storage 62 as imaging status information D6.

[0169] The battery 60 supplies power to each part of the imaging device 50A. The battery 60 can be a primary or secondary battery; for example, a lithium polymer battery can be used.

[0170] The storage device 62 stores control programs and image processing programs P1 for various calculations, as well as various data including structural images D1 (e.g., visible light images or infrared images) of the structure OBJ to be inspected. The storage device 62 also stores information D7 indicating the imaging plan and information D6 indicating the imaging status. For example, the storage device 62 can be a device including flash memory such as eMMC or SSD.

[0171] The communication interface 64 is a means for wireless communication with the controller 70. While the wireless communication method is not particularly limited, it is possible to use 2.4GHz or 5.7GHz wireless communication for communication with the controller 70. Furthermore, when communicating with the user terminal 300, wireless LAN or similar technologies can be used. Note that the communication method is not limited to wireless communication; wired communication can also be used.

[0172] The controller 70 may be a dedicated Proportional Control System (PROPO) transmitter, or it may be a terminal with a control application installed (for example, a tablet device).

[0173] The structural images D1 classified and organized by the imaging device 50A can be downloaded to a user terminal 300 (for example, a personal computer, a general-purpose computer such as a workstation, or a tablet terminal) for viewing. This makes it possible to perform various diagnostic and inspection tasks using the structural images D1, as well as create reports of the diagnostic results. The controller 70 and the user terminal 300 can also be used interchangeably.

[0174] According to Modification 1, by downloading the structural images D1 classified and organized by the imaging device 50A to the user terminal 300, it becomes possible to easily perform inspection work on the structural OBJ using the structural images D1.

[0175] (In combination with the fifth embodiment) The case in which member identification according to the fifth embodiment is performed in the imaging device 50A according to Modification 1 will be described.

[0176] As described above, in the fifth embodiment, the member discrimination unit 520 of the control unit 52 discriminates the members shown in the structural image D1 based on the correspondence between the information D7 regarding the imaging plan of the structural image D1 and the structural model D5.

[0177] In this case, in order to capture the structural image D1 according to the imaging plan information D7, it is preferable that the control unit 52 controls the system to capture the image only after it has determined that the detection results of the imaging position and imaging direction by the positioning unit 58 have reached the imaging plan information D7, that is, when the detection results of the imaging position and imaging direction by the positioning unit 58 match the imaging plan information D7.

[0178] As described above, by performing imaging control, the accuracy of component identification using information D7 related to the imaging plan can be improved.

[0179] In this embodiment, an example of mounting the camera 54 on an unmanned aerial vehicle has been described, but the present invention is not limited to this. For example, a camera mounted on any mobile body such as a vehicle or robot may be used. If an operator is on board the mobile body and operates it, the controller 70 is not required. It is also possible to use a camera that is not mounted on a mobile body.

[0180] [Differentiation 2] Furthermore, unlike the first modified example, it is also possible to configure a cloud server (hereinafter referred to as the cloud server) to have the image processing functions according to each of the above embodiments, and to upload the structure image D1 captured by the imaging device 50 to the cloud server.

[0181] Figure 21 is a block diagram showing a cloud server equipped with image processing capabilities according to Modification Example 2.

[0182] The cloud server 200 in Modification 2 is a server that can be used via a network such as the Internet. The cloud server 200 can communicate with the imaging device 50 and the user terminal 300 (for example, a general-purpose computer such as a personal computer or workstation, or a tablet terminal, etc.) via wired or wireless communication.

[0183] Furthermore, the cloud server 200 according to Modification 2 includes the image processing functions according to each of the above embodiments and is an example of an image processing apparatus according to the present invention.

[0184] Furthermore, the structural image D1 captured by the imaging device 50 may be uploaded to the cloud server 200 via, for example, a user terminal 300 or other external device. In this case, a communication connection between the cloud server 200 and the imaging device 50 is not required.

[0185] As shown in Figure 21, the processor 202 of the cloud server 200 can realize the functions of the component discrimination unit 220 and the identification information assignment unit 222 by reading and executing the image processing program P1 from the storage 204. Note that the component discrimination unit 220 and the identification information assignment unit 222 are the same as the component discrimination unit 120 and the identification information assignment unit 122 in each of the above embodiments, so their explanation is omitted.

[0186] The storage device 204 stores control programs and image processing programs P1 for various calculations, as well as various data including structural images D1 (e.g., visible light images or infrared images) of the structure OBJ to be inspected. The storage device 204 also stores information D7 indicating the imaging plan and information D6 indicating the imaging status. As the storage device 204, for example, a device including a magnetic disk such as an HDD, or a device including flash memory such as eMMC or SSD can be used.

[0187] The structural images D1, classified and organized on the cloud server 200, can be downloaded to the user terminal 300 for viewing. This makes it possible to perform various diagnostic and inspection tasks using the structural images D1, as well as create reports of the diagnostic results.

[0188] According to Modification 2, by downloading the structure images D1, which have been classified and organized on the cloud server 200, to the user terminal 300, it becomes possible to easily perform inspection work on the structure OBJ using the structure images D1.

[0189] [Difference 3] Furthermore, instead of the IaaS (Infrastructure as a Service) form as in Modification Example 2, it is also possible to provide the image processing functions according to each embodiment described above as SaaS (Software as a Service).

[0190] In this case, first, the user terminal 300 acquires the structure image D1 from the imaging device 50. Then, the user terminal 300 uses image processing functions for image classification and organization provided via the network NW from a cloud server 200 installed by an application service provider (ASP), etc., to classify and organize the structure image D1.

[0191] Here, when executing the image processing function provided by the cloud server 200, the structure image D1 may be uploaded to the cloud server 200, or the image processing function may be applied while it remains stored in the storage of the user terminal 300.

[0192] According to Modification 3, by classifying and organizing the structural image D1 using the image processing function provided by the cloud server 200, it becomes possible to easily perform inspection work on the structural OBJ using the structural image D1.

[0193] In the modified example 3, the imaging device 50 may call the image classification and organization function provided by the cloud server 200 to classify and organize the images in the storage of the imaging device 50.

[0194] Furthermore, the imaging device 50A and cloud server 200 according to the modified examples 1 to 3 can also be configured with the additional embodiment shown in Figure 19. [Explanation of Symbols]

[0195] 10…Image processing device, 12…Control unit, 14…Input unit, 16…Display unit, 18…Storage, 20…Communication I / F, 120…Component discrimination unit, 120A…Component discrimination AI, 120B, 120C…Image search engine, 120D, 120E…Image target calculation unit, 122…Identification information assignment unit, 124…Damage information assignment unit, 126…Assigned information editing unit, 128…Structural image search unit, 50, 50A…Imaging device, 52…Control unit, 54…Camera, 56…Drive unit, 58…Positioning unit, 60…Battery, 62…Storage, 64…Communication I / F, 66…Propeller, 520…Component discrimination unit, 522…Identification information assignment unit, 70…Controller, 200…Cloud server, 202…Processor, 204…Storage, 220…Component discrimination unit, 222…Identification information assignment unit, 300…User terminal

Claims

1. A member discrimination unit that discriminates members that are captured in an image of a structure to be discriminated against, The system includes an identification information assigning unit that assigns member identification information indicating the member identified by the member identification unit to the structural image to be identified, The member discrimination unit searches for reference structural images that show the same member as the member shown in the structural image to be discriminated, and discriminates the member shown in the structural image to be discriminated based on the reference structural images that were found.

2. A member discrimination unit that discriminates members that are captured in an image of a structure to be discriminated against, The system includes an identification information assigning unit that assigns member identification information indicating the member identified by the member identification unit to the structural image to be identified, The aforementioned member discrimination unit is A structural model including information showing the structure of the said structure, wherein a structural model associated with member identification information of the said structure and a reference structural image associated with the structural model are acquired, Image recognition is performed on the reference structure image and the structure image to be identified, and based on the results of the image recognition, the reference structure image containing the same member as the member shown in the structure image to be identified is searched for. An image processing device that identifies a component shown in an image of a structure to be identified based on the correspondence between a reference image of a structure that shows the same component as the component shown in the image of the structure to be identified and the structure model.

3. A member discrimination unit that discriminates members that are captured in an image of a structure to be discriminated against, The system includes an identification information assigning unit that assigns member identification information indicating the member identified by the member identification unit to the structural image to be identified, The member discrimination unit is an image processing device that acquires reference information for discriminating members that are captured in an image of a structure to be discriminated, which includes a structural model containing information indicating the structure of the structure, and discriminates members that are captured in an image of a structure to be discriminated based on the reference information.

4. A member discrimination unit that discriminates members that are captured in an image of a structure to be discriminated against, The system includes an identification information assigning unit that assigns member identification information indicating the member identified by the member identification unit to the structural image to be identified, The member discrimination unit acquires reference information including a structural model that includes information indicating the structure of the structure, and information regarding the imaging conditions that includes information on the imaging position and imaging direction when the structural image to be discriminated is taken, and discriminates the member that is captured in the structural image to be discriminated based on the correspondence between the structural model and the information regarding the imaging conditions.

5. A member discrimination unit that discriminates members that are captured in an image of a structure to be discriminated against, The system includes an identification information assigning unit that assigns member identification information indicating the member identified by the member identification unit to the structural image to be identified, The member discrimination unit acquires reference information including information on the imaging plan of the structure, which includes the imaging sequence of the structural images to be discriminated and information indicating the correspondence between the structural images and the members constituting the structure, and discriminates the members depicted in the structural images to be discriminated based on the correspondence between the imaging plan information and the structural images to be discriminated.

6. The information regarding the imaging plan includes information indicating the imaging position and imaging direction when the structure is imaged. The aforementioned image processing device is An imaging unit for imaging the aforementioned structure, A positioning unit that measures the imaging position and imaging direction of the imaging unit, A control unit controls the imaging position and imaging direction of the imaging unit so that they match the information indicating the imaging position and imaging direction included in the information regarding the imaging plan, The image processing apparatus according to claim 5, comprising:

7. The image processing apparatus according to any one of claims 1 to 6, wherein the identification information assigning unit assigns member identification information relating to the plurality of members to the structural image to be determined when the structural image to be determined includes a plurality of members.

8. The image processing apparatus according to any one of claims 1 to 6, further comprising a damage information application unit that detects damage from the image of the structure and applies information regarding the damage to the image of the structure.

9. The image processing apparatus according to any one of claims 1 to 6, further comprising an information editing unit for editing information assigned to the aforementioned structural image.

10. The image processing apparatus according to any one of claims 1 to 6, further comprising a structure image search unit that searches the structure image based on assigned information assigned to the structure image.

11. A step of identifying a member that is captured in an image of a structure to be identified, The process includes the step of adding member identification information to the image of the structure to be identified, which indicates a member identified as being present in the image of the structure to be identified. The step of identifying the member includes searching for a reference structural image in which the same member as the member shown in the structural image to be identified is shown, and then identifying the member shown in the structural image to be identified based on the reference structural image that was found.

12. A step of identifying a member that is captured in an image of a structure to be identified, The process includes the step of adding member identification information to the image of the structure to be identified, which indicates a member identified as being present in the image of the structure to be identified. The step of identifying the aforementioned member is: A step of obtaining a structural model that includes information indicating the structure of the said structure, the structural model associated with member identification information of the said structure, and a reference structural image associated with the structural model. The steps include performing image recognition on the reference structure image and the structure image to be identified, and based on the results of the image recognition, searching for the reference structure image that contains the same member as the member shown in the structure image to be identified, The steps include: determining the member shown in the structural image to be determined based on the correspondence between the reference structural image showing the same member as the member shown in the structural image to be determined and the structural model; Image processing methods, including those mentioned above.

13. A step of identifying a member that is captured in an image of a structure to be identified, The process includes the step of adding member identification information to the image of the structure to be identified, which indicates a member identified as being present in the image of the structure to be identified. The step of identifying the member includes obtaining reference information for identifying a member shown in the structural image of the target structure, which includes a structural model containing information indicating the structure of the structure, and identifying the member shown in the structural image of the target structure based on the reference information.

14. A step of identifying a member that is captured in an image of a structure to be identified, The process includes the step of adding member identification information to the image of the structure to be identified, which indicates a member identified as being present in the image of the structure to be identified. The step of identifying the member includes obtaining reference information including a structural model that includes information indicating the structure of the structure and information regarding the imaging conditions that includes information on the imaging position and imaging direction at the time of imaging of the structural image to be identified, and identifying the member shown in the structural image to be identified based on the correspondence between the structural model and the information regarding the imaging conditions.

15. A step of identifying a member that is captured in an image of a structure to be identified, The process includes the step of adding member identification information to the image of the structure to be identified, which indicates a member identified as being present in the image of the structure to be identified. The step of identifying the member includes obtaining reference information including the imaging plan of the structure, which includes the imaging order of the structural image to be identified and information indicating the correspondence between the members constituting the structure, and identifying the member shown in the structural image to be identified based on the correspondence between the imaging plan and the structural image to be identified.

16. The information relating to the imaging plan includes information indicating the imaging position and imaging direction when imaging the structure, The image processing method according to claim 15, wherein the imaging position and imaging direction of the imaging unit for imaging the structure match the information indicating the imaging position and imaging direction included in the information regarding the imaging plan.

17. A member discrimination function for discriminating members that are captured in an image of a structure to be discriminated against, An identification information assignment function that assigns member identification information indicating the member identified by the member identification function to the structural image of the object to be identified, An image processing program for a computer to implement, The aforementioned member discrimination function is an image processing program that searches for reference structural images in which the same member as the member shown in the structural image to be discriminated is shown, and then discriminates the member shown in the structural image to be discriminated based on the reference structural images that were found.

18. A member discrimination function for discriminating members that are captured in an image of a structure to be discriminated against, An identification information assignment function that assigns member identification information indicating the member identified by the member identification function to the structural image of the object to be identified, An image processing program for a computer to implement, The aforementioned member discrimination function is, A structural model including information showing the structure of the said structure, comprising a structural model associated with member identification information of the said structure, and a function for acquiring a reference structural image associated with the structural model, The function includes performing image recognition on the reference structure image and the structure image to be identified, and based on the results of the image recognition, searching for the reference structure image that contains the same member as the member shown in the structure image to be identified. A function to identify a component shown in the structural image to be identified based on the correspondence between the structural model and a reference structural image showing the same component as the component shown in the structural image to be identified, An image processing program that includes [this].

19. A member discrimination function for discriminating members that are captured in an image of a structure to be discriminated against, An identification information assignment function that assigns member identification information indicating the member identified by the member identification function to the structural image of the object to be identified, An image processing program for a computer to implement, The member discrimination function is an image processing program that obtains reference information for discriminating members that are captured in an image of a structure to be discriminated, which includes a structural model containing information indicating the structure of the structure, and discriminates members that are captured in an image of a structure to be discriminated based on the reference information.

20. A member discrimination function for discriminating members that are captured in an image of a structure to be discriminated against, An identification information assignment function that assigns member identification information indicating the member identified by the member identification function to the structural image of the object to be identified, An image processing program for a computer to implement, The member discrimination function is an image processing program that obtains reference information including a structural model containing information indicating the structure of the structure and information regarding the imaging conditions, including information on the imaging position and imaging direction at the time of imaging of the structural image to be discriminated, and discriminates the members shown in the structural image to be discriminated based on the correspondence between the structural model and the information regarding the imaging conditions.

21. A member discrimination function for discriminating members that are captured in an image of a structure to be discriminated against, An identification information assignment function that assigns member identification information indicating the member identified by the member identification function to the structural image of the object to be identified, An image processing program for a computer to implement, The member discrimination function is an image processing program that obtains reference information including the imaging plan of the structure, which includes the imaging order of the structural images to be discriminated and information indicating the correspondence between the structural images and the members constituting the structure, and discriminates the members depicted in the structural images to be discriminated based on the correspondence between the imaging plan information and the structural images to be discriminated.

22. The information relating to the imaging plan includes information indicating the imaging position and imaging direction when imaging the structure, The image processing program according to claim 21, wherein the imaging position and imaging direction of the imaging unit for imaging the structure match the information indicating the imaging position and imaging direction included in the information regarding the imaging plan.

Images