Image analysis device, image analysis method, and computer program

Abstract

This image analysis device comprises: a posture calculation unit that calculates a plurality of posture information items indicating the postures of a plurality of surfaces constituting the surface of a three-dimensional model that is disposed in a virtual space and represents an object in a real space; a reference calculation unit that calculates, on the basis of three or more surface position information items indicating the positions of three or more specific locations on the three-dimensional model, reference surface information indicating a reference surface; and a relative posture calculation unit that calculates, on the basis of the posture information items and the reference surface information, relative posture information indicating the posture of a surface relative to the reference surface.

Description

Image analysis device, image analysis method, and computer program

[0001] The present disclosure relates to an image analysis device, an image analysis method, and a computer program.

[0002] In the point cloud data utilization system described in Patent Document 1, an operator uses an operation unit to specify a predetermined number of points (for example, three points) among the projection coordinate points in an image of a region where an inspection target is displayed. Next, the control unit generates a virtual reference plane including each coordinate point corresponding to the specified projection coordinate point. The virtual reference plane virtually represents the surface in an ideal state without damage or the like. Next, the control unit extracts each coordinate point whose normal distance from the virtual reference plane is equal to or greater than a predetermined value as a feature point. Next, the display unit displays the projection coordinate points corresponding to the feature points in a distinguished manner by changing only the color of the projection coordinate points corresponding to the feature points. As a result, portions that protrude or are recessed by a predetermined amount or more from the virtual reference plane are characterized and displayed.

[0003] Japanese Patent Application Laid-Open No. 2016-10508

[0004] However, in the point cloud data utilization system described in Patent Document 1, when the protrusion amount or depression amount of the inspection target is relatively small, the normal distance from the virtual reference plane to the coordinate point may be less than the predetermined value, and the coordinate point may not be extracted as a feature point. That is, there is a possibility that a detection omission of an abnormality of the inspection target may occur.

[0005] Therefore, the present disclosure has been made in view of the above problems, and an object thereof is to provide an image analysis device, an image analysis method, and a computer program capable of suppressing the occurrence of a detection omission of an abnormality of an object.

[0006] According to this disclosure, an image analysis device is provided, comprising: a posture calculation unit that calculates a plurality of posture information indicating the posture of each of a plurality of surfaces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; a reference plane calculation unit that calculates reference plane information indicating a reference plane based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and a relative posture calculation unit that calculates relative posture information indicating the posture of the surface with respect to the reference plane based on the posture information and the reference plane information.

[0007] Furthermore, the present disclosure provides an image analysis method that includes the steps of: calculating a plurality of orientation information indicating the orientation of each of the plurality of surfaces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; calculating a reference surface information indicating a reference surface based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and calculating relative orientation information indicating the orientation of the surface with respect to the reference surface based on the orientation information and the reference surface information.

[0008] Furthermore, the present disclosure provides a computer program that causes a computer to perform the following steps: calculate a plurality of orientation information indicating the orientation of each of a plurality of faces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; calculate reference plane information indicating a reference plane based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and calculate relative orientation information indicating the orientation of the face with respect to the reference plane based on the orientation information and the reference plane information.

[0009] This disclosure provides an image analysis device, an image analysis method, and a computer program that can suppress the occurrence of missed detections of abnormalities in objects.

[0010] This is a block diagram showing an example configuration of an image analysis system according to one embodiment of the present invention. (a) and (b) are schematic cross-sectional views showing the phenomenon of delamination occurring in the concrete of a reinforced concrete structure, which is the object of study. This is a block diagram showing an example configuration of a server according to the same embodiment. This is a perspective view showing an example of a three-dimensional model according to the same embodiment. This is a perspective view showing an example of a surface constituting the surface of a three-dimensional model according to the same embodiment. This is a perspective view showing an example of a point cloud and a reference plane according to the same embodiment. This is a diagram showing a cross-section of an object superimposed on the point cloud and reference plane of a three-dimensional model according to the same embodiment. This is a perspective view showing a reference plane, a reference plane vector, a surface, a normal vector, and a reference vector according to the same embodiment. This is a perspective view showing an example of multiple surfaces sharing a single point in the point cloud according to the same embodiment. This is a diagram showing a color table according to the same embodiment. This is a diagram schematically showing a colored three-dimensional model according to the same embodiment. This is a flowchart showing an image analysis method according to the same embodiment.

[0011] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0012] Figure 1 is a diagram showing an example configuration of an image analysis system SYS according to one embodiment of the present invention. As shown in Figure 1, the image analysis system SYS includes a server 1. Server 1 corresponds to an example of the "image analysis device" in this disclosure.

[0013] Server 1 assists in detecting anomalies in an object existing in real space (hereinafter referred to as "object 100"), as an example. In other words, Server 1 assists in inspecting object 100. Object 100 is, for example, a reinforced concrete structure. In this case, for example, a reinforced concrete structure has walls (surfaces) that are substantially parallel to the vertical direction. Also, for example, an anomaly in object 100 is the delamination of the concrete.

[0014] Figures 2(a) and 2(b) are schematic cross-sectional views illustrating the phenomenon of delamination occurring in the concrete 101 of a reinforced concrete structure, which is the object 100. As shown in Figure 2(a), the reinforced concrete structure, which is the object 100, includes concrete 101 and reinforcing bars 103. The surface 104 of the concrete 101 is exposed. Direction D indicates the vertically upward direction. The reinforcing bars 103 are, for example, deformed reinforcing bars.

[0015] The reinforcing bars 103 may corrode and expand due to the penetration of water and other substances into the concrete 101. As a result, cracks 104 may occur in the concrete 101, starting from the reinforcing bars 103. Then, as shown in Figure 2(b), a portion of the concrete 101 may be pushed out, resulting in a delamination 102.

[0016] Server 1 assists in detecting the delamination portion 102 of the concrete 101 based on a three-dimensional model of the object 100. In particular, in this embodiment, Server 1 can suppress missed detections of the delamination portion 102 based on the three-dimensional model of the object 100, even when the delamination distance L1 of the delamination portion 102 is relatively small. The delamination distance L1 is, for example, about 1 mm to about 2 mm. Further details will be described later.

[0017] Returning to Figure 1, the image analysis system SYS comprises at least one terminal 2, at least one mobile device 3, at least one imaging device 5, or at least one three-dimensional measuring device 7.

[0018] Server 1, terminal 2, mobile device 3, imaging device 5, and three-dimensional measurement device 7 are connected to a network NW. The network NW includes, for example, the Internet, a private network, a public telephone network, a LAN (Local Area Network), and a short-range wireless network.

[0019] Terminal 2 is, for example, a personal computer (e.g., a laptop computer, a desktop computer, or a tablet).

[0020] The mobile device 3 is, for example, an unmanned mobile device or a manned mobile device. An unmanned mobile device is, for example, an unmanned aerial vehicle such as a drone, an unmanned ground vehicle, an unmanned submersible, or an unmanned surface vessel. An unmanned ground vehicle is, for example, an unmanned ground vehicle or an unmanned ground vehicle that mimics a living organism (for example, a snake-shaped unmanned ground vehicle). A manned mobile device is, for example, an aircraft, an automobile, a ship, or a submarine. The mobile device 3 includes a camera 4.

[0021] The imaging device 5 includes a camera 6. The imaging device 5 is, for example, a mobile device such as a smartphone. The imaging device 5 may also be, for example, the camera 6 itself.

[0022] Cameras 4 and 6 capture images of the object 100 and generate video data showing a video containing images of the object 100. The video is a collection of consecutive two-dimensional images. Cameras 4 and 6 may also generate multiple still image data, each showing multiple still images containing images of the object 100. The still images are two-dimensional images.

[0023] Hereafter, video data will be referred to as "video data 511" (Figure 2). Similarly, multiple still image data will be referred to as "still image data set 512" (Figure 2).

[0024] In the following, unless it is necessary to distinguish between cameras 4 and 6, cameras 4 and 6 will be collectively referred to as "camera CM".

[0025] The three-dimensional measuring device 7 measures the shape of the object 100 and generates three-dimensional data (hereinafter referred to as "three-dimensional data 513") representing the shape of the object 100. The three-dimensional data 513 is typically point cloud data. A three-dimensional model of the object 100 is formed from the three-dimensional data 513. The three-dimensional measuring device 7 may be contact-type or non-contact-type. Furthermore, the three-dimensional measuring device 7 may be active-type or passive-type. Active-type methods include, for example, optical radar (ToF (Time of Flight) method), active stereo method, or optical interferometry. The optical radar method measures the shape of the object 100 by irradiating the object 100 with light and measuring the time or phase change until the reflected light returns to the detector. An example of optical radar is LiDAR (Light Detection And Ranging). The active stereo method measures the shape of the object 100 by projecting a laser beam, slit light, or a code pattern of light. Optical interferometry measures the shape of an object 100 by irradiating it with light and utilizing the interference of the light. Passive methods include, for example, lens focusing.

[0026] Furthermore, the objects 100 to be imaged by the camera CM and the objects 100 to be measured by the three-dimensional measuring device 7 are not particularly limited, as long as they can be imaged by the camera CM or measured by the three-dimensional measuring device 7. Also, for example, the size, shape, pattern, and color of the objects 100 are not particularly limited. For example, the objects 100 are one or more movable or immovable property. The objects 100 are, for example, one or more objects. Typically, the objects 100 are one or more stationary objects. Stationary objects are, for example, man-made or natural objects. Man-made objects are, for example, structures, machinery, electronic equipment, or copyrighted works. Structures are, for example, buildings or infrastructure facilities. Buildings are, for example, office buildings or houses. Infrastructure facilities are facilities for developing social infrastructure. For example, infrastructure facilities are roads, bridges, road traffic facilities, power generation facilities, power transmission facilities, water treatment facilities, or gas distribution facilities. Machines include, for example, automobiles, work vehicles, trains, aircraft, ships, submarines, or robots. Natural objects include, for example, trees, forests, the ground, cliffs, coastlines, or rivers.

[0027] Terminal 2 acquires video data 511 or still image data set 512 generated by camera CM, or three-dimensional data 513 generated by three-dimensional measuring device 7. For example, terminal 2 receives video data 511 or still image data set 512 transmitted from camera CM, or three-dimensional data 513 transmitted from three-dimensional measuring device 7, via network NW. Terminal 2 transmits video data 511, still image data set 512, and three-dimensional data 513 to server 1 via network NW. The mobile device 3 and imaging device 5 may also transmit video data 511 or still image data set 512 to server 1 via network NW. The three-dimensional measuring device 7 may also transmit three-dimensional data 513 to server 1 via network NW.

[0028] Figure 3 is a block diagram showing an example configuration of Server 1 in Figure 1. As shown in Figure 3, Server 1 includes a calculation unit 10, a communication unit 40, and a storage unit 50. Server 1 may also include an input unit 20 and a display unit 30.

[0029] The input unit 20 is an input device for inputting various types of information to the calculation unit 10. For example, the input unit 20 may be a keyboard and pointing device, or a touch panel.

[0030] The display unit 30 displays various information. The display unit 30 is, for example, a liquid crystal display or an organic electroluminescent display.

[0031] The communication unit 40 is connected to a network NW. The communication unit 40 communicates with external devices connected to the network NW. The external devices are, for example, a terminal 2, a mobile device 3, an imaging device 5, and a three-dimensional measuring device 7. The communication unit 40 is a communication device that performs communication according to a predetermined communication protocol, and includes, for example, a network interface controller. The predetermined communication protocol is, for example, a protocol compliant with Ethernet® and an Internet Protocol Suite.

[0032] The communication unit 40 receives video data 511 or still image data set 512 from the terminal 2, mobile device 3, and imaging device 5 via the network NW. The communication unit 40 also receives three-dimensional data 513 from the terminal 2 and three-dimensional measuring device 7 via the network NW.

[0033] The storage unit 50 includes a storage device and stores data and computer programs. The storage unit 50 includes a main storage device such as a semiconductor memory and an auxiliary storage device such as a semiconductor memory and a hard disk drive. The storage unit 50 may also include a removable medium such as an optical disc. The storage unit 50 may be, for example, a non-temporary computer-readable storage medium.

[0034] The storage unit 50 stores video data 511, still image data set 512, and three-dimensional data 513. The video data 511, still image data set 512, and three-dimensional data 513 are associated with attribute information (hereinafter referred to as "attribute information AT"). Attribute information AT includes, for example, user information, data acquisition conditions, and object information. User information includes, for example, identification information of the user of terminal 2, mobile device 3, imaging device 5, or three-dimensional measuring device 7. Data acquisition conditions include, for example, imaging time or measurement time. Imaging time and measurement time are indicated by one or more of year, month, day, and time. Data acquisition conditions may also include information on imaging location or measurement location. Imaging location and measurement location are indicated, for example, by the position coordinates of camera CM and three-dimensional measuring device 7 acquired by GPS (Global Positioning System) or GNSS (Global Navigation Satellite System), respectively. Object information includes, for example, identification information of object 100. The object information may include the coordinates of a ground control point (GCP).

[0035] Furthermore, the memory unit 50 stores the three-dimensional model 514, posture information 515, reference plane information 516, relative posture information 517, and color table TB. This data is stored when it is generated. Details of this data will be described later.

[0036] The arithmetic unit 10 performs various calculations. The arithmetic unit 10 includes processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

[0037] Specifically, the calculation unit 10 includes a model generation unit 11, a posture calculation unit 13, a reference calculation unit 14, and a relative posture calculation unit 15. The calculation unit 10 may also include a reception unit 12, a coloring unit 16, and a display control unit 17. For example, the calculation unit 10 functions as the model generation unit 11, posture calculation unit 13, reference calculation unit 14, relative posture calculation unit 15, reception unit 12, coloring unit 16, and display control unit 17 by executing a computer program stored in the storage unit 50.

[0038] The model generation unit 11 obtains multiple two-dimensional images generated by imaging the object 100 from multiple different imaging positions using video data 511 or still image data set 512 from the storage unit 50. Then, the model generation unit 11 generates a three-dimensional model 514 of the object 100 based on the multiple two-dimensional images. The three-dimensional model 514 is placed in a virtual space. The three-dimensional model 514 shows the three-dimensional shape of the object 100. The three-dimensional model 514 is composed of point cloud data. Point cloud data is data that represents a point cloud. A point cloud is a collection of multiple points. Point cloud data includes the three-dimensional coordinates of each point. Point cloud data may further include one or more of the following information: color information (e.g., RGB values), normal vector information for each point, and reflectance information for each point.

[0039] As an example, the model generation unit 11 generates a three-dimensional model 514 by performing SfM (Structure from Motion) processing. SfM processing is a process that generates a three-dimensional model 514 by using the principle of triangulation based on multiple two-dimensional images generated by capturing an object 100 having multiple feature points from multiple imaging positions. SfM processing preferably includes bundle adjustment. Bundle adjustment is a process that minimizes reprojection errors. In addition to SfM processing, the model generation unit 11 may also perform MVS (Multi View Stereo) processing. MVS processing is a process that calculates the depth and normal for each pixel of each two-dimensional image by multi-view stereo measurement, integrates these, and generates a dense point cloud of the object 100.

[0040] Furthermore, the model generation unit 11 assigns surfaces to the three-dimensional model 514 based on the point cloud data. The surface of the three-dimensional model 514 is composed of multiple surfaces. A surface is typically a plane. A surface can be, for example, a polygon. A polygon can be, for example, a triangle. Each surface constituting the surface of the three-dimensional model 514 may be described as a "surface element" or "planar element".

[0041] The process of assigning faces by the model generation unit 11 is, for example, a process of converting point cloud data constituting the three-dimensional model 514 into mesh data. In this case, for example, the model generation unit 11 generates multiple polygonal faces (for example, triangles) by connecting points in the point cloud data, and represents the three-dimensional model 514 with these multiple polygonal faces. In this case, for example, the model generation unit 11 generates TIN (Triangulated Irregular Network) data based on the point cloud data, and represents the three-dimensional model 514 with the TIN data. As a result, the three-dimensional model 514 is represented by a set of triangular faces. Note that the process of assigning faces may also be, for example, a process of converting point cloud data into surface data.

[0042] Furthermore, the model generation unit 11 may add material information to the surfaces of the three-dimensional model 514 based on the two-dimensional images. The material information is information including the color and pattern of the object 100. For example, the model generation unit 11 may perform a process of mapping a texture to each surface (e.g., each polygon) constituting the mesh data of the three-dimensional model 514.

[0043] Note that the three-dimensional data 513 generated by the three-dimensional measuring device 7 is the same as the point cloud data constituting the three-dimensional model 514 generated by the model generation unit 11. Similarly, a surface may be assigned to the three-dimensional model represented by the three-dimensional data 513, and material information may be added to the surface.

[0044] The storage unit 50 stores the three-dimensional model 514 generated by the model generation unit 11. Also, the three-dimensional data 513 generated by the three-dimensional measuring device 7 may be treated as the three-dimensional model 514.

[0045] Note that, as methods for generating the three-dimensional data 513 and the three-dimensional model 514, the method executed by the three-dimensional measuring device 7 and the SfM process were exemplified. However, as long as the three-dimensional data or the three-dimensional model can be generated, these generation methods are not particularly limited. For example, the three-dimensional data or the three-dimensional model may be generated by 3D Gaussian Splatting or NeRF (Neural Radiance Field).

[0046] Next, the reception unit 12 will be described with reference to FIGS. 2, FIGS. 3 and FIG. 4. FIG. 4 is a perspective view showing an example of the three-dimensional model 514. As shown in FIG. 4, the three-dimensional model 514 is arranged in the virtual space VS. The three-dimensional model 514 represents the object 100 existing in the real space. In the virtual space VS, a three-dimensional coordinate system CS is set. The three-dimensional coordinate system CS is defined by an X-axis, a Y-axis, and a Z-axis that are orthogonal to each other. The three-dimensional coordinate system CS may be a coordinate system having a predetermined position in the virtual space VS as the origin, or may be a coordinate system to which geospatial coordinates (coordinates on the ground) or actual dimension information is assigned.

[0047] As shown in FIGS. 3 and 4, the reception unit 12 receives the designation of a target area 530 to be processed among the three-dimensional model 514. For example, the display control unit 17 causes the three-dimensional model 514 to be displayed on the display unit of the terminal 2 in response to a request from the terminal 2. The user of the terminal 2 operates the input unit of the terminal 2 to designate the target area 530 on the three-dimensional model 514 displayed on the display unit 30. As a result, the terminal 2 transmits information indicating the target area 530 to the server 1 via the network NW. The communication unit 40 of the server 1 receives the information indicating the target area 530. Then, the reception unit 12 receives the designation of the target area 530 via the communication unit 40. The storage unit 50 stores the information indicating the target area 530. The configurations of the display unit and the input unit of the terminal 2 are the same as those of the display unit 30 and the input unit 20 of the server 1, respectively.

[0048] Next, the posture calculation unit 13 will be described with reference to FIGS. 3 and 5. FIG. 5 is a perspective view showing a surface 525 constituting the three-dimensional model 514. In FIG. 5, a part of the three-dimensional model 514 is shown. The surface of the three-dimensional model 514 is composed of a plurality of surfaces 525. In the example of FIG. 5, the surface 525 is a triangular surface (for example, a polygon). Each surface 525 is formed by connecting points 519 that constitute a point group 518.

[0049] As shown in FIGS. 3 and 5, the posture calculation unit 13 calculates a plurality of posture information 515 indicating the respective postures of a plurality of surfaces 525 within the target area 530 (FIG. 4). The storage unit 50 stores the posture information 515. In the example of FIG. 5, the posture information 515 is normal line information indicating the normal line of the surface 525. Therefore, according to the present embodiment, the posture of the surface 525 can be accurately represented by the normal line. Specifically, the normal line information includes information on the normal vector 535 of the surface 525. Thus, as a preferred example, the posture calculation unit 13 calculates the normal vector 535 for each of the plurality of surfaces 525. The normal vector 535 is, for example, a unit normal vector. Specifically, the posture calculation unit 13 calculates the normal vector 535 based on the three-dimensional coordinates of a plurality of points 519 located at the vertices of the surface 525. For example, the start point and the end point of the normal vector 535 are indicated by three-dimensional coordinates.

[0050] Next, the reference calculation unit 14 will be described with reference to Figures 2, 3, and 6. Figure 6 is a perspective view showing an example of a point cloud 518 and a reference plane 520. In Figure 6, a portion of the point cloud 518 that constitutes the three-dimensional model 514 is shown.

[0051] As shown in Figures 3 and 6, the reference calculation unit 14 calculates reference surface information 516 indicating a reference surface 520 based on three or more surface position pieces indicating the positions of three or more specific locations 521 on the three-dimensional model 514. The reference surface 520 is a plane. The reference surface 520 is, for example, a virtual surface. The storage unit 50 stores the reference surface information 516. The surface position pieces are typically three-dimensional coordinates in the three-dimensional coordinate system CS. In the example in Figure 5, the number of specific locations 521 is "6", but it is not particularly limited as long as it is three or more. The specific locations 521 are locations in the three-dimensional model 514 that correspond to locations where no abnormality has occurred in the object 100 (normal locations).

[0052] Specifically, the reference calculation unit 14 calculates reference plane information 516 indicating the reference plane 520 by least squares method based on the three-dimensional coordinates of three or more points 519, each of which is located at three or more specific locations 521. The reference plane information 516 includes, for example, information indicating the equation of a plane. The reference plane 520 is a least squares plane. According to this embodiment, by performing the least squares method on the three-dimensional coordinates of three or more points 519, a reference plane 520 fitted to multiple points 519 can be determined with high accuracy.

[0053] More specifically, the reception unit 12 receives the designation of specific locations 521. For example, the display control unit 17 displays the three-dimensional model 514 on the display unit of terminal 2 in response to a request from terminal 2. The user of terminal 2 operates the input unit of terminal 2 to designate three or more specific locations 521 within the target area 530 (Figure 4) of the three-dimensional model 514 displayed on the display unit 30. For example, the user operates the cursor displayed on the display unit 30 with a pointing device and designates three or more specific locations 521 by clicking on three or more locations within the target area 530. In this case, locations within the target area 530 where no abnormalities have occurred (i.e., normal locations) are designated as specific locations 521. For example, if the object 100 is a concrete structure, normal locations of the concrete 101 (Figure 2) where no delamination, spalling, or peeling has occurred are designated.

[0054] Terminal 2 transmits information indicating the designated specific location 521 to Server 1 via the network NW. The communication unit 40 of Server 1 receives the information indicating the specific location 521. Then, the reception unit 12 accepts the designation of the specific location 521 via the communication unit 40. In other words, the reception unit 12 accepts the designation of three or more specific locations 521 via an input unit operated by the user. Therefore, according to this embodiment, the server 1 does not need to perform processing to designate the specific location 521, and the processing load can be reduced. The storage unit 50 stores information indicating three or more specific locations 521.

[0055] The input section of terminal 2 corresponds to an example of the "input device" in this disclosure.

[0056] The reference calculation unit 14 calculates reference surface information 516 indicating the reference surface 520 based on three or more points 519 located at three or more specific locations 521 received by the reception unit 12. In other words, the reference calculation unit 14 calculates the reference surface information 516 based on points 519 located at specific locations 521 where no abnormality has occurred (normal specific locations 521). Therefore, the reference surface 520 is suitable as a reference for detecting abnormalities in the object 100.

[0057] For example, the reference calculation unit 14 may consider the point 519 closest to the specific location 521 among a plurality of points 519 as the point 519 located at the specific location 521. Alternatively, for example, the reference calculation unit 14 may consider the point 519 located within a circle or sphere of a predetermined radius centered on the specific location 521 as the point 519 located at the specific location 521.

[0058] Next, the relative attitude calculation unit 15 will be explained with reference to Figures 3 and 7. Figure 7 is a diagram showing the cross-section of the object 100 superimposed on the point cloud 518 and reference plane 520 of the three-dimensional model 514. The object 100 in Figure 7 is, as an example, the object 100 shown in Figure 2(b). Note that in Figure 7, the dot hatching indicating the concrete 101 has been omitted for the sake of simplifying the drawing. Hereafter, we will focus on the boundary region 540 between the floating portion 102 of the concrete 101 and the non-floating portion of the concrete 101. Also in Figure 7, in the boundary region 540, a plane 525 is shown with points 519a and 519b of the point cloud 518 as two of its three vertices.

[0059] As shown in Figures 3 and 7, the relative posture calculation unit 15 calculates relative posture information 517 that indicates the posture of the surface 525 with respect to the reference surface 520. Relative posture information 517 is calculated for each surface 525. As an example, the relative posture information 517 includes information indicating the inclination of the normal vector 535 with respect to the reference surface 520. Therefore, according to this embodiment, the presence or absence of the floating portion 102 can be determined based on the inclination of the normal vector 535 with respect to the reference surface 520. As a result, even when the floating distance L1 of the floating portion 102 is relatively small, the occurrence of detection failures of the floating portion 102 can be suppressed.

[0060] In contrast, in the comparative example (for example, Patent Document 1), if the lifting distance L1 of the floating portion 102 is relatively small, there is a possibility of missing detection of the floating portion 102. In other words, in the comparative example, point 519, where the normal distance L2 from the reference surface 520 to point 519 is greater than or equal to a predetermined value, is extracted as a feature point. Therefore, if the lifting distance L1 is relatively small, depending on the setting of the reference surface 520, the normal distance L2 to point 519, which indicates the surface of the floating portion 102, may be less than the predetermined value. As a result, point 519, which indicates the surface of the floating portion 102, may not be extracted as a feature point, and there is a possibility of missing detection of the floating portion 102. In other words, in the comparative example, the accuracy of setting the reference surface 520 has a large impact on the accuracy of detecting the floating portion 102.

[0061] In contrast, in this embodiment, the presence or absence of the floating portion 102 can be determined based on the inclination of the normal vector 535 with respect to the reference surface 520. Therefore, compared to the comparative example, the accuracy of setting the reference surface 520 has less influence on the accuracy of detecting the floating portion 102. Thus, compared to the comparative example, the occurrence of missed detections of the floating portion 102 can be suppressed, and setting the reference surface 520 is easier.

[0062] Next, the reference calculation unit 14 and the relative attitude calculation unit 15 will be described in detail with reference to Figures 3 and 8. Figure 8 is a perspective view showing the reference plane 520, reference plane vector 560, plane 525, normal vector 535, and reference vector 550. Note that in Figure 8, for the sake of simplicity, one plane 525, one normal vector 535, and one reference vector 550 are shown.

[0063] As shown in Figure 8, the reference calculation unit 14 sets a reference vector 550 relative to the normal vector 535. The reference vector 550 is parallel to the reference plane 520. The reference vector 550 is, for example, a unit vector. The start and end points of the reference vector 550 are indicated by three-dimensional coordinates. The reference plane information 516 (Figure 3) includes information on the reference vector 550.

[0064] Specifically, the reference calculation unit 14 determines the reference plane vector 560. The reference plane vector 560 is parallel to the reference plane 520 and is a vector included in the reference plane 520. For example, the reference plane vector 560 is represented by three-dimensional coordinates. The reference plane information 516 (Figure 3) includes information on the reference plane vector 560. The reference plane vector 560 is, for example, a unit vector.

[0065] For example, the reference plane vector 560 is parallel to the intersection line of the XY plane and the reference plane 520 in the three-dimensional coordinate system CS, the intersection line of the YZ plane and the reference plane 520, or the intersection line of the ZX plane and the reference plane 520. An intersection line is a straight line formed at the intersection of two planes. In this case, for example, the X and Y axes in the three-dimensional coordinate system CS are parallel to the horizontal direction, and the Z axis is parallel to the vertical direction. Also, the XY plane is the plane containing the X and Y axes. The YZ plane is the plane containing the Y and Z axes. The ZX plane is the plane containing the Z and X axes.

[0066] For example, when inspecting vertical changes in the surface 104 of an object 100, the reference calculation unit 14 sets a vector parallel to the intersection of the YZ plane and the reference plane 520, or a vector parallel to the intersection of the ZX plane and the reference plane 520, from among multiple vectors within the reference plane 520, as the reference plane vector 560. For example, when inspecting horizontal changes in the surface 104 of an object 100, the reference calculation unit 14 sets a vector parallel to the intersection of the XY plane and the reference plane 520, from among multiple vectors within the reference plane 520, as the reference plane vector 560.

[0067] As described above, the reference calculation unit 14 can determine the reference surface vector 560 from among a plurality of vectors within the reference surface 520 according to the purpose of the inspection.

[0068] The reference calculation unit 14 translates the reference plane vector 560 so that the starting point of the reference plane vector 560 coincides with the starting point of the normal vector 535. The reference plane vector 560 after translation is the reference vector 550. In this way, the reference vector 550 is set with respect to the normal vector 535.

[0069] The reference plane vector 560 is information indicating the characteristics (e.g., orientation) of the reference plane 520. Therefore, the reference vector 550 is also information indicating the characteristics (e.g., orientation) of the reference plane 520. Furthermore, the reference vector 550 is parallel to the reference plane vector 560. Therefore, for example, the reference vector 550 is parallel to the intersection line of the XY plane and the reference plane 520, the intersection line of the YZ plane and the reference plane 520, or the intersection line of the ZX plane and the reference plane 520 in the three-dimensional coordinate system CS.

[0070] The reference calculation unit 14 sets a reference vector 550 for each of the normal vectors 535 of all surfaces 525 within the target region 530 (Figure 4).

[0071] The relative attitude calculation unit 15 calculates the dot product (hereinafter referred to as "dot product value IP1") of the normal vector 535 and the reference vector 550 for each surface 525 within the target region 530. The dot product value IP1 indicates the slope of the normal vector 535 with respect to the reference vector 550. In other words, the dot product value IP1 indicates the slope of the normal vector 535 with respect to the reference surface 520. The dot product value IP1 is an example of relative attitude information 517. The storage unit 50 stores relative attitude information 517 that includes information indicating the dot product value IP1.

[0072] As described above, according to this embodiment, the relative posture calculation unit 15 can easily calculate the inclination of the normal vector 535 with respect to the reference plane 520, that is, the inclination of the surface 525 with respect to the reference plane 520, by calculating the dot product value IP1 of the normal vector 535 and the reference vector 550. For example, the larger the absolute value of the dot product value IP1, the greater the inclination of the normal vector 535 (surface 525) with respect to the reference plane 520. Therefore, the relative posture calculation unit 15 can estimate that the larger the absolute value of the dot product value IP1, the higher the probability that an abnormality has occurred in the region of the object 100 corresponding to the surface 525 for which the dot product value IP1 was calculated. For example, the relative posture calculation unit 15 may also calculate the angle between the reference vector 550 and the normal vector 535. In this case, the angle is an example of relative posture information 517.

[0073] As described above with reference to Figures 7 and 8, according to this embodiment, the relative posture calculation unit 15 calculates relative posture information 517, which indicates the posture of the surface 525 relative to the reference surface 520, based on the posture information 515 (normal vector 535) of each surface 525 within the target region 530 and the reference surface information 516 (reference vector 550) which indicates the characteristics of the reference surface 520. Therefore, based on the relative posture information 517, abnormalities in the object 100 (for example, floating portion 102) can be detected. As a result, in this embodiment, compared to the comparative example described above, even when the degree of abnormality in the object 100 (for example, floating distance L1) is relatively small, the occurrence of missed detection of abnormalities can be suppressed. Furthermore, in this embodiment, since the presence or absence of abnormalities in the object 100 can be determined based on the relative posture information 517, the influence of the accuracy of setting the reference surface 520 on the accuracy of abnormality detection is smaller compared to the comparative example described above. Therefore, setting the reference surface 520 is easier compared to the comparative example.

[0074] Furthermore, in this embodiment, the relative posture information 517 (for example, the absolute value of the dot product IP1) indicates the degree of difference between the posture of surface 525 and the posture of the reference surface 520. For example, a large difference between the posture of surface 525 and the posture of the reference surface 520 indicates that an abnormality has occurred in the area of ​​the object 100 corresponding to surface 525. For example, if there is no or small difference between the posture of surface 525 and the posture of the reference surface 520, it indicates that the area of ​​the object 100 corresponding to surface 525 is normal.

[0075] Next, the coloring section 16 will be described with reference to Figures 3 and 9 to 11. The coloring section 16 shown in Figure 3 colors the three-dimensional model 514 based on a plurality of relative orientation information 517 for each of the plurality of surfaces 525 that constitute the surface of the three-dimensional model 514. The display control unit 17 displays the colored three-dimensional model 514 on the terminal 2. Therefore, the user can recognize the relative orientation information 517 by the color when looking at the three-dimensional model 514. As a result, according to this embodiment, the user can easily estimate that an abnormality has occurred in the area that is colored to indicate a large difference between the orientation of the reference surface 520 and the orientation of the surface 525.

[0076] Specifically, the coloring unit 16 colors the three-dimensional model 514 based on the dot product value IP1 indicated by the relative posture information 517. An example of this will be explained with reference to Figures 9 to 11.

[0077] Figure 9 is a perspective view showing an example of multiple faces 525 sharing a single point 519. In the example in Figure 9, the single point 519 forms the vertices of six faces 525. The relative attitude calculation unit 15 calculates the average value (hereinafter, "dot product value IP2") of multiple dot product values ​​IP1 calculated for the multiple faces 525 sharing the single point 519. The relative attitude calculation unit 15 then associates the dot product value IP2 with the point 519 shared by the multiple faces 525. The dot product value IP2 is an example of relative attitude information 517. The relative attitude calculation unit 15 calculates the dot product value IP2 for each point 519 and associates the dot product value IP2 with each point 519.

[0078] The coloring unit 16 then colors each point 519 according to the range to which the dot product value IP2 associated with each point 519 belongs.

[0079] The relative attitude calculation unit 15 may also calculate the average value of multiple normal vectors 535 of multiple surfaces 525. In this case, the relative attitude calculation unit 15 calculates the dot product of the average value of the normal vectors 535 and the reference vector 550 (Figure 8). In this case, the relative attitude calculation unit 15 associates the dot product value IP2 with point 519.

[0080] Figure 10 shows the color table TB referenced by the coloring unit 16. As shown in Figure 10, the color table TB associates multiple ranges R1 to R10 with multiple color information C1 to C10. Ranges R1 to R10 represent the range of the dot product value IP2 associated with point 519. Color information C1 to C10 represent different colors from each other. Color information C1 to C10 are represented, for example, by RGB values. The color table TB is stored in the storage unit 50 in advance. Note that the number and values ​​of ranges and color information are examples and are not particularly limited. In addition, ranges have positive and negative signs, but ranges of absolute values ​​may also be set.

[0081] The coloring unit 16 refers to the color table TB and colors each point 519 of the point cloud 518 according to the color information corresponding to the range to which the dot product value IP2 associated with each point 519 belongs. For example, if the dot product value IP2 belongs to range R2, the coloring unit 16 colors the point 519 with the color indicated by the color information C2.

[0082] Preferably, the colored portion 16 colors the three-dimensional model 514 in such a way that it can distinguish between cases where the normal vector 535 of the surface 525 is pointing vertically upward and vertically downward with respect to the horizontal direction. According to this preferred example, by looking at the colored three-dimensional model 514, the user can easily estimate areas where a part of the object 100 shows signs of peeling or flaking (for example, the floating portion 102 in Figure 2(b)). Vertically upward and vertically downward include directions oblique to the horizontal direction.

[0083] The color table TB in this preferred example will be described in detail. Figure 10 shows the reference plane 520, the normal vector 535, and the reference vector 550. In the example of Figure 10, the reference plane 520 is parallel to the ZX plane. The reference vector 550 is parallel to the Z axis and points vertically upward. The reference vector 550 is parallel to the intersection line of the YZ plane and the reference plane 520. The Z axis is also oriented vertically. The X and Y axes are oriented horizontally. The normal vector 535a points vertically downward with respect to the horizontal. The normal vector 535b points vertically upward with respect to the horizontal.

[0084] If the normal vector 535a points vertically downward, the dot product value IP2 associated with point 519 located at a vertex of face 525 will have, for example, a negative sign. On the other hand, if the normal vector 535b points vertically upward, the dot product value IP2 associated with point 519 located at a vertex of face 525 will have, for example, a positive sign.

[0085] In the color table TB, the color information C1 to C5 associated with ranges R1 to R5 when the dot product value IP2 has a negative sign is distinguishable from the color information C6 to C10 associated with ranges R6 to R10 when the dot product value IP2 has a positive sign. In this case, for example, the color information C1 to C5 associated with ranges R1 to R5 includes colors belonging to either the cool color system or the warm color system. Also, for example, the color information C6 to C10 associated with ranges R6 to R10 includes colors belonging to the other of the cool color system or the warm color system.

[0086] Figure 11 is a schematic diagram showing a colored three-dimensional model 514. As shown in Figure 11, the display control unit 17 displays the colored three-dimensional model 514 on the terminal 2. In Figure 11, for ease of understanding, the color information C1 to C10 and the coordinate axes are shown. The display control unit 17 may also display the color information C1 to C10 on the terminal 2. In the example of Figure 11, the three-dimensional model 514 is colored by the color information C1 and C5. The user can infer that an abnormality has occurred in the area colored by the color information C1. For example, the user can infer that a floating portion 102 (Figure 2(b)) has occurred in the area colored by the color information C1. This is because the color information C1 corresponds to the dot product value IP2 belonging to range R1.

[0087] As described above with reference to Figures 10 and 11, according to this embodiment, the coloring unit 16 colors the three-dimensional model 514 according to a plurality of relative orientation information 517 (dot product value IP2) associated with a plurality of points 519 of the point cloud 518. Therefore, by looking at the three-dimensional model 514, the user can recognize the relative orientation information 517 by color. As a result, the user can estimate the area where an anomaly has occurred in the object 100 by the color indicating an anomaly. In particular, since the reference vector 550 is parallel to the intersection line of the YZ plane and the reference plane 520, the user can easily recognize vertical changes in the surface of the three-dimensional model 514 (surface 104 of the object 100) by color.

[0088] Here, as another example, if the reference vector 550 is parallel to the intersection line of the XY plane and the reference plane 520, the user can easily recognize the horizontal changes in the surface of the three-dimensional model 514 (the surface 104 of the object 100) by color. In this case, for example, the colored part 16 colors the three-dimensional model 514 in a way that distinguishes between the case where the normal vector 535 is pointing in the vertical direction and the case where it is pointing in the horizontal direction. The vertical direction and the horizontal direction include directions oblique to the vertical or horizontal direction. In this case, for example, the vertical direction is when the angle is greater than or equal to a threshold angle and less than or equal to 90 degrees relative to the horizontal direction, and the horizontal direction is when the angle is less than a threshold angle relative to the horizontal direction. The threshold angle is, for example, 45 degrees, but is not particularly limited.

[0089] Next, an image analysis method according to this embodiment will be described with reference to Figures 3 and 12. Figure 12 is a flowchart of the image analysis method. The image analysis method is executed by the server 1. As shown in Figure 12, the image analysis method includes steps S1 to S10. The computer program stored in the storage unit 50 causes the arithmetic unit 10 to execute steps S1 to S10. In other words, the computer program product realizes steps S1 to S10 when the computer program is executed by the arithmetic unit 10. The arithmetic unit 10 corresponds to an example of a "computer" in this disclosure.

[0090] As shown in Figures 3 and 12, first, in step S1, the model generation unit 11 acquires a three-dimensional model 514. Specifically, the model generation unit 11 generates a three-dimensional model 514 representing the object 100 based on multiple two-dimensional images generated by imaging the object 100 from multiple different imaging positions. Alternatively, the model generation unit 11 sets three-dimensional data 513 based on the measurement results of the object 100 into the three-dimensional model 514.

[0091] Next, in step S2, the display control unit 17 causes the three-dimensional model 514 to be displayed on the terminal 2.

[0092] Next, in step S3, the reception unit 12 receives a designation of a target area 530 within the three-dimensional model 514 from the terminal 2 via the communication unit 40.

[0093] Next, in step S4, the attitude calculation unit 13 calculates a plurality of attitude information 515 that indicates the attitude of each of the plurality of surfaces 525 that constitute the surface in the target region 530 of the three-dimensional model 514.

[0094] Next, in step S5, the display control unit 17 causes the target area 530 of the three-dimensional model 514 to be displayed on the terminal 2.

[0095] Next, in step S6, the reception unit 12 receives a designation of three or more specific locations 521 in the target area 530 from the terminal 2 via the communication unit 40.

[0096] Next, in step S7, the reference calculation unit 14 calculates reference surface information 516 indicating the reference surface 520 based on three or more surface position pieces indicating the respective positions of three or more specific locations 521. Specifically, the reference calculation unit 14 calculates the reference surface information 516 using the least squares method.

[0097] Next, in step S8, the relative posture calculation unit 15 calculates relative posture information 517 for each surface 525, indicating the posture of the surface 525 with respect to the reference surface 520, based on the posture information 515 and the reference surface information 516.

[0098] Next, in step S9, the coloring unit 16 colors the three-dimensional model 514 based on a plurality of relative posture information 517.

[0099] Next, in step S10, the display control unit 17 displays the colored three-dimensional model 514 on the terminal 2. Then the image analysis method is completed.

[0100] As described above with reference to Figure 12, according to the image analysis method of this embodiment, the relative posture calculation unit 15 calculates relative posture information 517 that indicates the posture of the surface 525 with respect to the reference surface 520 (step S8). Therefore, abnormalities in the object 100 can be detected based on the relative posture information 517. As a result, compared to the comparative example described above, even when the degree of abnormality in the object 100 is relatively small, the occurrence of missed detection of abnormalities can be suppressed.

[0101] The image analysis method according to this embodiment is particularly effective when the object 100 is a reinforced concrete structure. For example, as shown in Figure 7, even when the lifting distance L1 of the floating portion 102 is relatively small, the detection of the floating portion 102 can be suppressed.

[0102] (First Modification) In the first modification of this embodiment, the relative posture calculation unit 15 performs threshold processing. Specifically, the relative posture calculation unit 15 determines that an area of ​​the object 100, indicated by the surface 525 corresponding to the dot product value IP1 or the point 519 corresponding to the dot product value IP2, is abnormal if the absolute value of the dot product value IP1 or the absolute value of the dot product value IP2 indicated by the relative posture information 517 is greater than or equal to a threshold.

[0103] Furthermore, for example, the relative posture calculation unit 15 may provide multiple thresholds. In this case, for example, the relative posture calculation unit 15 determines that the area of ​​the object 100 indicated by the surface 525 corresponding to the dot product value IP1 or the point 519 corresponding to the dot product value IP2 is abnormal if the absolute value of the dot product value IP1 or the absolute value of the dot product value IP2 is greater than or equal to the first threshold. Also, for example, the relative posture calculation unit 15 determines that the area of ​​the object 100 indicated by the surface 525 corresponding to the dot product value IP1 or the point 519 corresponding to the dot product value IP2 is quasi-abnormal if the absolute value of the dot product value IP1 or the absolute value of the dot product value IP2 is greater than or equal to the second threshold and less than the first threshold. The second threshold is smaller than the first threshold. Quasi-abnormal means a state that is not abnormal, but is closer to abnormal than normal. Note that the multiple thresholds may be three or more thresholds.

[0104] (Second Modification) The second modification of this embodiment differs from the above embodiment, in that the coloring unit 16 colors each surface 525, while the coloring unit 16 colors each point 519. That is, in the second modification, in the color table TB of Figure 10, the ranges R6 to R10 represent the ranges of the dot product value IP1 of each surface 525. Therefore, the coloring unit 16 refers to the color table TB of the second modification and colors each surface 525 according to the color information corresponding to the range to which the dot product value IP1 for each surface 525 belongs.

[0105] Preferred embodiments and modifications of the present disclosure have been described in detail above with reference to the attached drawings, but the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations can be conceived within the scope of the technical idea set forth in the claims, and these too are understood to fall within the technical scope of the present disclosure.

[0106] The apparatus or system described herein may be implemented as a single apparatus, or it may be implemented by a plurality of apparatuses (e.g., a cloud server) that are partially or completely connected by a network. For example, some or all of the model generation unit 11, reception unit 12, posture calculation unit 13, reference calculation unit 14, relative posture calculation unit 15, coloring unit 16, and display control unit 17 may be implemented by the same computer or server. For example, the model generation unit 11, reception unit 12, posture calculation unit 13, reference calculation unit 14, relative posture calculation unit 15, coloring unit 16, and display control unit 17 may each be implemented by separate computers or servers. Alternatively, for example, the model generation unit 11, reception unit 12, posture calculation unit 13, reference calculation unit 14, relative posture calculation unit 15, coloring unit 16, and display control unit 17 may be implemented by a terminal 2 or a three-dimensional measuring device 7. For example, video data 511, still image data set 512, three-dimensional data 513, three-dimensional model 514, posture information 515, reference plane information 516, relative posture information 517, and color table TB may each be stored in separate storage devices or servers.

[0107] The series of processes performed by the apparatus described herein may be implemented using software, hardware, or a combination of software and hardware. Computer programs for implementing each function of the arithmetic unit 10 according to this embodiment can be created and implemented on a PC or the like. Furthermore, a computer-readable recording medium containing such a computer program can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, and flash memory. Alternatively, the computer program may be distributed without using a recording medium, for example, via a network.

[0108] Furthermore, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. For example, in Figure 12, step S4 may be executed between steps S7 and S8. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0109] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that are obvious to those skilled in the art from the description herein, in addition to or instead of the effects described herein.

[0110] Furthermore, the following configurations also fall within the technical scope of this disclosure.

[0111] (Item 1) An image analysis device comprising: an orientation calculation unit that calculates a plurality of orientation information indicating the orientation of each of the plurality of surfaces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; a reference plane calculation unit that calculates reference plane information indicating a reference plane based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and a relative orientation calculation unit that calculates relative orientation information indicating the orientation of the surface with respect to the reference plane based on the orientation information and the reference plane information.

[0112] (Item 2) The image analysis device described in Item 1, wherein the orientation information is normal information indicating the normal of the surface.

[0113] (Item 3) The image analysis device according to Item 2, wherein the normal information includes information on the normal vector of the surface, and the relative orientation information includes information indicating the inclination of the normal vector with respect to the reference surface.

[0114] (Item 4) The image analysis device according to Item 3, wherein the reference plane information includes information on a reference vector parallel to the reference plane, and the relative orientation information includes information indicating the dot product of the normal vector and the reference vector.

[0115] (Item 5) An image analysis device according to any one of Items 1 to 4, further comprising a coloring unit for coloring the three-dimensional model based on a plurality of relative pose information.

[0116] (Item 6) The image analysis device according to Item 5, wherein the orientation information includes information on the normal vector of the surface, and the coloring portion colors the three-dimensional model in a way that distinguishes between the case where the normal vector is pointed vertically upward and the case where it is pointed vertically downward with respect to the horizontal direction.

[0117] (Item 7) The image analysis device according to Item 5, wherein the orientation information includes information on the normal vector of the surface, and the coloring portion colors the three-dimensional model in a way that distinguishes between the case where the normal vector is pointing in the vertical direction and the case where it is pointing in the horizontal direction.

[0118] (Item 8) An image analysis device according to any one of Items 1 to 7, further comprising a reception unit that receives the designation of three or more specific locations via an input device operated by a user.

[0119] (Item 9) The image analysis device according to any one of Items 1 to 8, wherein the reference calculation unit calculates the reference surface information by the least squares method based on the three or more surface position information.

[0120] (Item 10) The object is a reinforced concrete structure, and the image analysis device is one of the items described in items 1 to 9.

[0121] (Item 11) An image analysis method comprising: a step of calculating a plurality of orientation information indicating the orientation of each of the plurality of surfaces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; a step of calculating a reference surface information indicating a reference surface based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and a step of calculating relative orientation information indicating the orientation of the surface with respect to the reference surface based on the orientation information and the reference surface information.

[0122] (Item 12) A computer program that causes a computer to perform the following steps: calculate a plurality of orientation information indicating the orientation of each of the plurality of surfaces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; calculate reference surface information indicating a reference plane based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and calculate relative orientation information indicating the orientation of the surface with respect to the reference plane based on the orientation information and the reference surface information.

[0123] This disclosure provides an image analysis device, an image analysis method, and a computer program, and has industrial applicability.

[0124] 1 Server (image analysis device), 11 Model generation unit, 12 Reception unit, 13 Pose calculation unit, 14 Reference calculation unit, 15 Relative pose calculation unit, 16 Coloring unit, 17 Display control unit

Claims

1. An image analysis device comprising: a posture calculation unit that calculates a plurality of posture information indicating the posture of each of the plurality of surfaces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; a reference plane calculation unit that calculates reference plane information indicating a reference plane based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and a relative posture calculation unit that calculates relative posture information indicating the posture of the surface with respect to the reference plane based on the posture information and the reference plane information.

2. The image analysis device according to claim 1, wherein the orientation information is normal information indicating the normal of the surface.

3. The image analysis apparatus according to claim 2, wherein the normal information includes information on the normal vector of the surface, and the relative orientation information includes information indicating the inclination of the normal vector with respect to the reference surface.

4. The image analysis apparatus according to claim 3, wherein the reference plane information includes information on a reference vector parallel to the reference plane, and the relative orientation information includes information indicating the dot product of the normal vector and the reference vector.

5. The image analysis apparatus according to claim 1, further comprising a coloring unit for coloring the three-dimensional model based on a plurality of relative pose information.

6. The image analysis apparatus according to claim 5, wherein the orientation information includes information on the normal vector of the surface, and the coloring portion colors the three-dimensional model in a way that distinguishes between cases where the normal vector is pointed vertically upward and cases where it is pointed vertically downward with respect to the horizontal direction.

7. The image analysis apparatus according to claim 5, wherein the orientation information includes information on the normal vector of the surface, and the coloring portion colors the three-dimensional model in a way that distinguishes between the case where the normal vector is oriented vertically and the case where it is oriented horizontally.

8. The image analysis apparatus according to claim 1 or 2, further comprising a reception unit that receives the designation of three or more specific locations via an input device operated by a user.

9. The image analysis apparatus according to claim 1 or 2, wherein the reference calculation unit calculates the reference surface information by the least squares method based on the three or more surface position information.

10. The image analysis device according to claim 1 or claim 2, wherein the object is a reinforced concrete structure.

11. An image analysis method comprising: calculating a plurality of orientation information indicating the orientation of each of the plurality of surfaces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; calculating reference surface information indicating a reference surface based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and calculating relative orientation information indicating the orientation of the surface with respect to the reference surface based on the orientation information and the reference surface information.

12. A computer program that causes a computer to perform the following steps: calculate a plurality of orientation information indicating the orientation of each of the plurality of faces constituting the surface of a three-dimensional model that is placed in a virtual space and represents an object in real space; calculate reference plane information indicating a reference plane based on three or more surface position information indicating the positions of three or more specific locations on the three-dimensional model; and calculate relative orientation information indicating the orientation of the face with respect to the reference plane based on the orientation information and the reference plane information.

Images