Image analysis device, image analysis method, and computer program

Abstract

An image analysis device disclosed herein comprises: an image generation unit that generates a first two-dimensional image and a second two-dimensional image; and a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image. The image generation unit generates a first two-dimensional image in which a first three-dimensional model generated on the basis of a measurement result of the shape of an object at a first time in a real space is represented as a two-dimensional plane from a first viewpoint in a first virtual space. The image generation unit generates a second two-dimensional image in which a second three-dimensional model generated on the basis of a measurement result of the shape of the object at a second time in the real space is represented as a two-dimensional plane from a second viewpoint in a second virtual space. The same coordinate system is set in the first virtual space and the second virtual space. In the coordinate system, the position and the line-of-sight direction of the second viewpoint are the same as the position and the line-of-sight direction of the first viewpoint, respectively.

Description

Image analysis device, image analysis method, and computer program

[0001] The present disclosure relates to an image analysis device, an image analysis method, and a computer program.

[0002] The difference detection device described in Patent Document 1 acquires a plurality of images by photographing a structure from a plurality of directions in the first inspection, and generates three-dimensional first point cloud data from the plurality of images. Then, in the second inspection, the difference detection device again acquires a plurality of images obtained by imaging the structure from a plurality of directions, and generates three-dimensional second point cloud data from the plurality of images. The first point cloud data and the second point cloud data indicate the same location of the structure.

[0003] The difference detection device compares the color information indicating the color of the points included in the first point cloud data with the color information indicating the color of the points included in the second point cloud data. Then, for a location where the difference between the two is equal to or greater than the threshold value, in the second point cloud data, the color of the location where the difference is equal to or greater than the threshold value is changed and displayed.

[0004] Japanese Patent Application Laid-Open No. 2018-181056

[0005] However, in the difference detection device described in Patent Document 1, the entire region of the three-dimensional data (the first point cloud data and the second point cloud data) is analyzed to calculate the difference. Therefore, the processing load of the difference detection device was excessive.

[0006] Therefore, the present disclosure has been made in view of the above problems, and an object thereof is to provide an image analysis device, an image analysis method, and a computer program that can reduce the processing load when calculating the difference.

[0007] According to this disclosure, an image analysis device is provided, comprising: an image generation unit that generates a first two-dimensional image and a second two-dimensional image; and a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image, wherein the image generation unit generates a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; and generates a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space, wherein the same coordinate system is set in the first virtual space and the second virtual space, and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0008] Furthermore, the present disclosure provides an image analysis method comprising the steps of: generating a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space; and calculating the difference between the first two-dimensional image and the second two-dimensional image, wherein the same coordinate system is set for the first and second virtual spaces, and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0009] Furthermore, according to this disclosure, a computer program is provided which causes a computer to perform the following steps: generate a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; generate a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space; and calculate the difference between the first two-dimensional image and the second two-dimensional image, wherein the same three-dimensional coordinate system is set in the first virtual space and the second virtual space, and in the three-dimensional coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0010] According to this disclosure, an image analysis device, an image analysis method, and a computer program can be provided that can reduce the processing load when calculating differences.

[0011] This is a block diagram showing an example configuration of an image analysis system according to one embodiment of the present invention. This is a block diagram showing an example configuration of a server according to the same embodiment. (a) is a schematic perspective view showing a first viewpoint, a first three-dimensional model, and a first rectangular prism in a first virtual space according to the same embodiment. (b) is a schematic perspective view showing a second viewpoint, a second three-dimensional model, and a second rectangular prism in a second virtual space according to the same embodiment. (a) is a schematic side view showing a first viewpoint, a first three-dimensional model, and a first rectangular prism in a first virtual space according to the same embodiment. (b) is a schematic side view showing a second viewpoint, a second three-dimensional model, and a second rectangular prism in a second virtual space according to the same embodiment. (a) is a diagram showing an example of a first two-dimensional image according to the same embodiment. (b) is a diagram showing an example of a second two-dimensional image according to the same embodiment. (c) is a diagram showing the concept of difference data according to the same embodiment. (a) is a diagram showing an example of a two-dimensional image including a region where the first two-dimensional image and the second two-dimensional image differ according to the same embodiment. (b) is a diagram showing an example of a three-dimensional model that includes a region where the first two-dimensional image and the second two-dimensional image differ according to the same embodiment. This is a flowchart of the image analysis method according to the same embodiment.

[0012] Preferred embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted. In this embodiment, a rectangular parallelepiped includes a cube.

[0013] Figure 1 is a diagram showing an example configuration of an image analysis system SYS according to one embodiment of the present invention. As shown in Figure 1, the image analysis system SYS includes a server 1. Server 1 corresponds to an example of the "image analysis device" in this disclosure.

[0014] Server 1 calculates the difference between two two-dimensional images, each representing the same three-dimensional model on a two-dimensional plane from the same viewpoint, in order to detect the differences between two three-dimensional models generated by measuring the same object at different times. In this way, according to this embodiment, since the difference between two-dimensional images is calculated, the processing load on Server 1 when calculating the difference can be reduced compared to the case where the entire area of ​​the three-dimensional data constituting the three-dimensional model is analyzed to calculate the difference. Details of this point will be described later.

[0015] Furthermore, the image analysis system SYS comprises at least one terminal 2, at least one mobile device 3, at least one imaging device 5, or at least one three-dimensional measuring device 7.

[0016] Server 1, terminal 2, mobile device 3, imaging device 5, and three-dimensional measurement device 7 are connected to a network NW. The network NW includes, for example, the Internet, a private network, a public telephone network, a LAN (Local Area Network), and a short-range wireless network.

[0017] Terminal 2 is, for example, a personal computer (e.g., a laptop computer, a desktop computer, or a tablet).

[0018] The mobile device 3 is, for example, an unmanned mobile device or a manned mobile device. An unmanned mobile device is, for example, an unmanned aerial vehicle such as a drone, an unmanned ground vehicle, an unmanned submersible, or an unmanned surface vessel. An unmanned ground vehicle is, for example, an unmanned ground vehicle or an unmanned ground vehicle that mimics a living organism (for example, a snake-shaped unmanned ground vehicle). A manned mobile device is, for example, an aircraft, an automobile, a ship, or a submarine. The mobile device 3 includes a camera 4.

[0019] The imaging device 5 includes a camera 6. The imaging device 5 is, for example, a mobile device such as a smartphone. The imaging device 5 may also be, for example, the camera 6 itself.

[0020] Cameras 4 and 6 capture images of the object and generate video data showing a video containing images of the object. The video is a collection of consecutive two-dimensional images. Cameras 4 and 6 may also generate multiple still image data, each showing multiple still images containing images of the object. The still images are two-dimensional images.

[0021] Hereafter, video data will be referred to as "video data 511" (Figure 2). Similarly, multiple still image data will be referred to as "still image data set 512" (Figure 2).

[0022] In the following, unless it is necessary to distinguish between cameras 4 and 6, cameras 4 and 6 will be collectively referred to as "camera CM".

[0023] The three-dimensional measuring device 7 measures the shape of an object and generates three-dimensional data (hereinafter referred to as "three-dimensional data 513") representing the shape of the object. The three-dimensional data 513 is typically point cloud data. A three-dimensional model of the object is formed from the three-dimensional data 513. The three-dimensional measuring device 7 may be contact-type or non-contact-type. Furthermore, the three-dimensional measuring device 7 may be active-type or passive-type. Active-type methods include, for example, optical radar (ToF (Time of Flight) method), active stereo method, or optical interferometry. Optical radar measures the shape of an object by irradiating it with light and measuring the time or phase change until the reflected light returns to the detector. An example of optical radar is LiDAR (Light Detection And Ranging). Active stereo method measures the shape of an object by projecting a laser beam, slit light, or a code pattern of light. Optical interferometry measures the shape of an object by irradiating it with light and utilizing the interference of the light. Passive methods include, for example, lens focusing.

[0024] Furthermore, the objects to be imaged by the camera CM and the objects to be measured by the three-dimensional measuring device 7 are not particularly limited, as long as they can be imaged by the camera CM or measured by the three-dimensional measuring device 7. Also, for example, the size, shape, pattern, and color of the objects are not particularly limited. For example, an object is one or more movable or immovable property. An object is, for example, one or more objects. Typically, an object is one or more stationary objects. Stationary objects are, for example, man-made or natural objects. Man-made objects are, for example, structures, machinery, electronic equipment, or copyrighted works. Structures are, for example, buildings or infrastructure facilities. Buildings are, for example, office buildings or houses. Infrastructure facilities are facilities for developing social infrastructure. For example, infrastructure facilities are roads, bridges, road traffic facilities, power generation facilities, power transmission facilities, water treatment facilities, or gas distribution facilities. Machinery is, for example, automobiles, work vehicles, trains, aircraft, ships, submarines, or robots. Natural objects include, for example, trees, forests, the ground surface, cliffs, coastlines, or rivers.

[0025] Terminal 2 acquires video data 511 or still image data set 512 generated by camera CM, or three-dimensional data 513 generated by three-dimensional measuring device 7. For example, terminal 2 receives video data 511 or still image data set 512 transmitted from camera CM, or three-dimensional data 513 transmitted from three-dimensional measuring device 7, via network NW. Terminal 2 transmits video data 511, still image data set 512, and three-dimensional data 513 to server 1 via network NW. The mobile device 3 and imaging device 5 may also transmit video data 511 or still image data set 512 to server 1 via network NW. The three-dimensional measuring device 7 may also transmit three-dimensional data 513 to server 1 via network NW.

[0026] Figure 2 is a block diagram showing an example configuration of Server 1 in Figure 1. As shown in Figure 2, Server 1 includes an arithmetic unit 10, a communication unit 40, and a storage unit 50. Server 1 may also include an input unit 20 and a display unit 30.

[0027] The input unit 20 is an input device for inputting various types of information to the calculation unit 10. For example, the input unit 20 may be a keyboard and pointing device, or a touch panel.

[0028] The display unit 30 displays various information. The display unit 30 is, for example, a liquid crystal display or an organic electroluminescent display.

[0029] The communication unit 40 is connected to a network NW. The communication unit 40 communicates with external devices connected to the network NW. The external devices are, for example, a terminal 2, a mobile device 3, an imaging device 5, and a three-dimensional measuring device 7. The communication unit 40 is a communication device that performs communication according to a predetermined communication protocol, and includes, for example, a network interface controller. The predetermined communication protocol is, for example, a protocol compliant with Ethernet® and an Internet Protocol Suite.

[0030] The communication unit 40 receives video data 511 or still image data set 512 from the terminal 2, mobile device 3, and imaging device 5 via the network NW. The communication unit 40 also receives three-dimensional data 513 from the terminal 2 and three-dimensional measuring device 7 via the network NW.

[0031] The storage unit 50 includes a storage device and stores data and computer programs. The storage unit 50 includes a main storage device such as a semiconductor memory and an auxiliary storage device such as a semiconductor memory and a hard disk drive. The storage unit 50 may also include a removable medium such as an optical disc. The storage unit 50 may be, for example, a non-temporary computer-readable storage medium.

[0032] The storage unit 50 stores video data 511, still image data set 512, and three-dimensional data 513. The video data 511, still image data set 512, and three-dimensional data 513 are associated with attribute information (hereinafter referred to as "attribute information AT"). Attribute information AT includes, for example, user information, data acquisition conditions, and object information. User information includes, for example, identification information of the user of terminal 2, mobile device 3, imaging device 5, or three-dimensional measuring device 7. Data acquisition conditions include, for example, imaging time or measurement time. Imaging time and measurement time are indicated by one or more of year, month, day, and time. Data acquisition conditions may also include information on imaging location or measurement location. Imaging location and measurement location are indicated, for example, by the position coordinates of camera CM and three-dimensional measuring device 7 acquired by GPS (Global Positioning System) or GNSS (Global Navigation Satellite System), respectively. Object information includes, for example, identification information of the object. The object information may include the coordinates of a ground control point (GCP).

[0033] Furthermore, the memory unit 50 stores the first three-dimensional model 521, the second three-dimensional model 522, the first two-dimensional image 110, the second two-dimensional image 120, the difference data 130 (130A), the two-dimensional image 140, and the three-dimensional model 140A. This data is stored when it is generated. Details of this data will be described later.

[0034] The arithmetic unit 10 performs various calculations. The arithmetic unit 10 includes processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

[0035] Specifically, the calculation unit 10 includes an image generation unit 12 and a difference calculation unit 14. The calculation unit 10 may also include a model generation unit 11, an image adjustment unit 13, an image enhancement unit 15, and a display control unit 16. For example, the calculation unit 10 functions as the model generation unit 11, the image generation unit 12, the image adjustment unit 13, the difference calculation unit 14, the image enhancement unit 15, and the display control unit 16 by executing a computer program stored in the storage unit 50.

[0036] The model generation unit 11 obtains multiple two-dimensional images generated by imaging an object from multiple different imaging positions from the video data 511 or still image dataset 512 of the storage unit 50. Then, the model generation unit 11 generates a three-dimensional model of the object based on the multiple two-dimensional images. The three-dimensional model is placed in a virtual space. The three-dimensional model shows the three-dimensional shape of the object. The three-dimensional model is composed of point cloud data. Point cloud data is data that represents a point cloud. A point cloud is a collection of multiple points. Point cloud data includes the three-dimensional coordinates of each point. Point cloud data may further include one or more of the following information: color information (e.g., RGB values) of each point, normal vector information of each point, and reflectance information.

[0037] As an example, the model generation unit 11 generates a three-dimensional model by performing SfM (Structure from Motion) processing. SfM processing is a process that generates a three-dimensional model by using the principle of triangulation based on multiple two-dimensional images generated by capturing an object having multiple feature points from multiple imaging positions. SfM processing preferably includes bundle adjustment. Bundle adjustment is a process that minimizes reprojection errors. In addition to SfM processing, the model generation unit 11 may also perform MVS (Multi View Stereo) processing. MVS processing is a process that calculates the depth and normal for each pixel of each two-dimensional image by multi-view stereo measurement, integrates these, and generates a dense point cloud of the object.

[0038] Furthermore, it is preferable for the model generation unit 11 to assign faces to the three-dimensional model based on the point cloud data. The process of assigning faces is, for example, a process of converting the point cloud data constituting the three-dimensional model into mesh data. In this case, for example, the model generation unit 11 generates a plurality of polygonal faces (for example, triangles) by connecting points in the point cloud data, and represents the three-dimensional model with the plurality of polygonal faces. In this case, for example, the model generation unit 11 generates TIN (Triangulated Irregular Network) data based on the point cloud data, and represents the three-dimensional model with the TIN data. As a result, the three-dimensional model is represented by a set of triangular faces. Note that the process of assigning faces may also be, for example, a process of converting point cloud data into surface data.

[0039] Furthermore, the model generation unit 11 may add material information to the surfaces assigned to the three-dimensional model based on a two-dimensional image. Material information includes information about the color and pattern of the object. For example, the model generation unit 11 may perform a process of mapping a texture to each surface (e.g., each polygon) that constitutes the mesh data of the three-dimensional model.

[0040] As explained above with reference to Figure 2, the three-dimensional model is generated based on the imaging results of the object by the camera CM. Therefore, imaging of the object by the camera CM can be considered as a measurement of the object's shape. Thus, the time spent imaging the object with the camera CM is the same as the time spent measuring the object with the camera CM.

[0041] The three-dimensional data 513 generated by the three-dimensional measuring device 7 is the same as the point cloud data that constitutes the three-dimensional model generated by the model generation unit 11. Similarly, surfaces may be assigned to the three-dimensional model shown by the three-dimensional data 513, and material information may be added to these surfaces.

[0042] The memory unit 50 stores the three-dimensional model generated by the model generation unit 11 as the first three-dimensional model 521. The memory unit 50 also stores another three-dimensional model generated by the model generation unit 11 as the second three-dimensional model 522. Alternatively, three-dimensional data 513 generated by the three-dimensional measurement device 7 may be treated as the first three-dimensional model 521, and another three-dimensional data 513 may be treated as the second three-dimensional model 522. These details will be described later.

[0043] The methods used to generate the three-dimensional data 513, the first three-dimensional model 521, and the second three-dimensional model 522 are exemplified by the methods performed by the three-dimensional measuring device 7 and SfM processing. However, these generation methods are not particularly limited as long as three-dimensional data or three-dimensional models can be generated. For example, the three-dimensional data or three-dimensional model may be generated by 3D Gaussian splatting or NeRF (Neural Radiance Field). Furthermore, the three-dimensional data or three-dimensional model is not limited to point cloud data, and the data format is not particularly limited.

[0044] Next, the image generation unit 12 will be described with reference to FIGS. 2 to 4. FIG. 3(a) is a perspective view schematically showing a first viewpoint 61, a first three-dimensional model 521, and a first rectangular parallelepiped 71 in the first virtual space VS1. FIG. 3(b) is a perspective view schematically showing a second viewpoint 81, a second three-dimensional model 522, and a second rectangular parallelepiped 72 in the second virtual space VS2. FIG. 4(a) is a side view schematically showing the first viewpoint 61, the first three-dimensional model 521, and the first rectangular parallelepiped 71 in the first virtual space VS1. FIG. 4(a) corresponds to FIG. 3(a). FIG. 4(b) is a side view schematically showing the second viewpoint 81, the second three-dimensional model 522, and the second rectangular parallelepiped 72 in the second virtual space VS2. FIG. 4(b) corresponds to FIG. 3(b). In FIGS. 3 and 4, a virtual reference plane 75 is shown for ease of understanding.

[0045] As shown in FIGS. 2 and 3(a), the first three-dimensional model 521 is arranged in the first virtual space VS1. The first three-dimensional model 521 is a solid model generated based on the measurement result of the shape of an object at a first time (hereinafter, "first time T1") in the real space. The first time T1 is indicated by the attribute information AT. For example, the first time T1 is indicated by one or more of year, month, day, and time.

[0046] A three-dimensional coordinate system CS is set in the first virtual space VS1. The three-dimensional coordinate system CS is defined by an X-axis, a Y-axis, and a Z-axis that are orthogonal to each other. The three-dimensional coordinate system CS may be a coordinate system having a predetermined position in the first virtual space VS1 as the origin O, or may be a coordinate system assigned with geospatial coordinates (coordinates on the ground) or actual dimension information. In FIGS. 3 and 4, as an example, for convenience of explanation, the origin O of the three-dimensional coordinate system CS is set on the virtual reference plane 75. The three-dimensional coordinate system CS corresponds to an example of the "coordinate system" of the present disclosure.

[0047] The image generation unit 12 generates a first two-dimensional image 110 that represents the first three-dimensional model 521 on a two-dimensional plane from the first viewpoint 61 of the first virtual space VS1. The first viewpoint 61 faces the line-of-sight direction 62 at the position p1 of the first virtual space VS1. In other words, the image generation unit 12 generates the first two-dimensional image 110 by capturing the first three-dimensional model 521 from the first viewpoint 61. Stated more specifically, the image generation unit 12 generates the first two-dimensional image 110 by pseudo-imaging the first three-dimensional model 521 with the first virtual camera 63 having the first viewpoint 61. The storage unit 50 stores the first two-dimensional image 110.

[0048] Specifically, the model generation unit 11 sets a first rectangular parallelepiped 71 for the first three-dimensional model 521 in the first virtual space VS1. The first rectangular parallelepiped 71 surrounds the first three-dimensional model 521. The first rectangular parallelepiped 71 has a shape and size corresponding to the shape and size of the first three-dimensional model 521. The first rectangular parallelepiped 71 is a rectangular parallelepiped that defines the display range when displaying the first three-dimensional model 521 on the viewer. Hereinafter, a rectangular parallelepiped that defines the display range when displaying a three-dimensional model on the viewer may be referred to as a "defining rectangular parallelepiped". The viewer is software or a device for viewing images. The defining rectangular parallelepiped is, for example, a bounding box. The first rectangular parallelepiped 71 corresponds to an example of the "rectangular parallelepiped" of the present disclosure.

[0049] Then, the first viewpoint 61 is set on the first rectangular parallelepiped 71. Therefore, according to the present embodiment, the first viewpoint 61 can be set more easily as compared with the case where the first viewpoint 61 is determined at an arbitrary point in the first virtual space VS1. In particular, since the first rectangular parallelepiped 71 is always set when displaying the first three-dimensional model 521 on the viewer, it is possible to suppress the occurrence of additional processing only for determining the first viewpoint 61.

[0050] Preferably, the first viewpoint 61 is set to any of the eight vertices a1, a2, a3, a4, a5, a6, a7, a8 of the first rectangular prism 71, any of the midpoints b1, b2, b3, b4, b5, b6, b7, b8, b9, b10, b11, b12 of the twelve edges of the first rectangular prism 71, or any of the centers c1, c2, c3, c4, c5, c6 of the six faces of the first rectangular prism 71. Thus, according to this preferred example, the first viewpoint 61 can be set more easily.

[0051] For example, if the first viewpoint 61 is set to one of the vertices a1 to a8, the line of sight 62 of the first viewpoint 61 will point in the direction of the diagonal of the first rectangular prism 71. In other words, the line of sight 62 will point to the vertex diagonally opposite to the vertex where the first viewpoint 61 is set.

[0052] If the first viewpoint 61 is set to one of the vertices a1 to a8, then, for example, a first two-dimensional image 110 can be generated as if the first three-dimensional model 521 were viewed from diagonally above or diagonally below. In particular, since the positions of vertices a1 to a8 are uniquely determined, the calculation process for the positions of vertices a1 to a8 can be omitted.

[0053] In the example shown in Figure 3(a), the position p1 of the first viewpoint 61 is vertex a1. Specifically, the first viewpoint 61 is set at vertex a1 of the first rectangular prism 71 and faces in the direction of the diagonal 73 extending from vertex a1. In other words, the line of sight 62 is directed along the diagonal 73, from vertex a1 to vertex a7.

[0054] For example, if the first viewpoint 61 is set to one of the midpoints b1 to b12, the line of sight direction 62 of the first viewpoint 61 will be directed diagonally opposite the midpoint where the first viewpoint 61 is set, across the first three-dimensional model 521. In this case, for example, if the first viewpoint 61 is set to midpoint b1, the line of sight direction 62 will be directed from midpoint b1 to midpoint b6.

[0055] If the first viewpoint 61 is set to one of the midpoints b1 to b12, for example, a first two-dimensional image 110 can be generated when viewing the first three-dimensional model 521 from diagonally above, diagonally below, or diagonally to the side.

[0056] For example, if the first viewpoint 61 is set to one of the centers c1 to c6, the line of sight direction 62 of the first viewpoint 61 will point toward the center opposite to the center where the first viewpoint 61 is set, in a direction perpendicular to the plane containing the center where the first viewpoint 61 is set. In this case, for example, if the first viewpoint 61 is set to center c1, the line of sight direction 62 will point from center c1 toward center c3.

[0057] If the first viewpoint 61 is set to one of the centers c1 to c6, for example, a first two-dimensional image 110 can be generated when viewing the first three-dimensional model 521 from a direction perpendicular to the face of the first rectangular parallelepiped 71.

[0058] On the other hand, as shown in Figure 3(b), the second three-dimensional model 522 is placed in the second virtual space VS2. The second three-dimensional model 522 is a three-dimensional model generated based on the measurement results of the shape of the object in real space at the second time (hereinafter, "second time T2"). The second time T2 is indicated by attribute information AT. For example, the second time T2 is indicated by one or more of the year, month, day, and time. The second time T2 is different from the first time T1. For example, the second time T2 is a time earlier than the first time T1.

[0059] The second virtual space VS2 is set to a three-dimensional coordinate system CS. In other words, the same three-dimensional coordinate system CS is set in both the second virtual space VS2 and the first virtual space VS1.

[0060] The image generation unit 12 generates a second two-dimensional image 120, which represents the second three-dimensional model 522 on a two-dimensional plane from a second viewpoint 81 in the second virtual space VS2. The second viewpoint 81 is oriented in the line of sight direction 82 at position p2 in the second virtual space VS2. In other words, the image generation unit 12 generates the second two-dimensional image 120 by capturing the second three-dimensional model 522 from the second viewpoint 81. To put it another way, the image generation unit 12 generates the second two-dimensional image 120 by simulating imaging the second three-dimensional model 522 with a second virtual camera 83 having the second viewpoint 81. The storage unit 50 stores the second two-dimensional image 120.

[0061] In particular, as shown in Figures 3(a) and 3(b), in the three-dimensional coordinate system CS, the position p2 and line of sight direction 82 of the second viewpoint 81 (second virtual camera 83) are the same as the position p1 and line of sight direction 62 of the first viewpoint 61 (first virtual camera 63), respectively.

[0062] Therefore, as shown in Figures 4(a) and 4(b), the three-dimensional coordinates (aX, aY, aZ) indicating the position p2 of the second viewpoint 81 are the same as the three-dimensional coordinates (aX, aY, aZ) indicating the position p1 of the first viewpoint 61. Also, the line-of-sight directions 62 and 82 (the orientations of the first virtual camera 63 and the second virtual camera 83) are represented by the same rotation angles (θx, θy, θz) with respect to the reference direction (reference orientation). θx represents the rotation angle around the X axis, θy represents the rotation angle around the Y axis, and θz represents the rotation angle around the Z axis.

[0063] Furthermore, the first three-dimensional model 521 and the second three-dimensional model 522 are three-dimensional models representing the same object. However, the time periods (first time T1 and second time T2) at which the shape of the object was measured are different for the first three-dimensional model 521 and the second three-dimensional model 522. Therefore, if part or all of the object changes over time, the form of the first three-dimensional model 521 and the form of the second three-dimensional model 522 representing the object may differ. The form of the object may be, for example, the shape, pattern, or color of part or all of the object, or a combination thereof.

[0064] In the examples in Figures 3 and 4, the upper half of the object (first three-dimensional model 521) at the first time T1 is gone in the object (second three-dimensional model 522) at the second time T2. However, for the part of the object that has not changed between the first time T1 and the second time T2 (for example, the bottom), the position of the corresponding part of the first three-dimensional model 521 (for example, the bottom 521a) and the position of the corresponding part of the second three-dimensional model 522 (for example, the bottom 522a) are the same in the three-dimensional coordinate system CS.

[0065] As shown in Figure 4(b), the model generation unit 11 sets up a second rectangular prism 72 for the second three-dimensional model 522 in the second virtual space VS2. The second rectangular prism 72 is a defined rectangular prism similar to the first rectangular prism 71. As shown in Figures 4(a) and 4(b), the centers of the defined rectangular prisms (center 74 of the first rectangular prism 71 and center 76 of the second rectangular prism 72) are located inside the three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522).

[0066] Next, the difference calculation unit 14 will be described with reference to Figures 2 and 5. Figure 5(a) is a diagram showing an example of the first two-dimensional image 110. Figure 5(b) is a diagram showing an example of the second two-dimensional image 120.

[0067] As shown in Figure 5(a), the first two-dimensional image 110 includes images 111 and 112. As shown in Figure 5(b), the second two-dimensional image 120 includes images 121 and 122.

[0068] Images 111 and 121 are images of the same object. Image 112 exists in the first two-dimensional image 110 but not in the second two-dimensional image 120. Image 122 does not exist in the first two-dimensional image 110 but does exist in the second two-dimensional image 120.

[0069] The difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120. The difference calculation unit 14 then outputs difference data 130 indicating the difference. The storage unit 50 stores the difference data 130. Figure 5(c) is a diagram illustrating the concept of difference data 130. As shown in Figure 5(c), the difference data 130 includes difference data 1120 (dense dots) corresponding to the image 112 in Figure 5(a), difference data 1220 (sparse dots) corresponding to the image 122 in Figure 5(b), and difference data 1000. Difference data 1000 is data indicating that there is no difference.

[0070] Specifically, the difference calculation unit 14 calculates the difference for each pixel of the first two-dimensional image 110 and the second two-dimensional image 120. In other words, the difference calculation unit 14 calculates the difference in pixel values ​​between corresponding pixels in the first two-dimensional image 110 and the second two-dimensional image 120. More specifically, one pixel is composed of N pixel elements, where N is an integer greater than or equal to 2. Therefore, the pixel value of one pixel contains the values ​​of N pixel elements. Thus, the difference calculation unit 14 calculates the difference in pixel element values ​​for each pixel between corresponding pixels. For this reason, the difference data 130 is a set of differences (difference values) in the pixel element values ​​of each pixel.

[0071] For example, N = 3. In this case, the N pixel elements that make up one pixel are, for example, an R (red) element, a G (green) element, and a B (blue) element. In this case, the three pixel element values ​​included in the pixel value of one pixel are the R value, the G value, and the B value. The R value, G value, and B value represent their respective brightness values. Therefore, the difference calculation unit 14 calculates the difference in R values, the difference in G values, and the difference in B values ​​between corresponding pixels in the first two-dimensional image 110 and the second two-dimensional image 120.

[0072] For example, the difference calculation unit 14 may calculate the difference by subtracting the second two-dimensional image 120 from the first two-dimensional image 110, or by subtracting the first two-dimensional image 110 from the second two-dimensional image 120.

[0073] Specifically, for example, the difference calculation unit 14 may calculate the difference by subtracting the corresponding pixel element value of the second two-dimensional image 120 from the pixel element value of the first two-dimensional image 110, or by subtracting the corresponding pixel element value of the first two-dimensional image 110 from the pixel element value of the second two-dimensional image 120.

[0074] Furthermore, the difference calculation unit 14 may perform the following processing on the difference data 130 to generate new difference data (hereinafter referred to as "difference data 130A").

[0075] Specifically, the difference calculation unit 14 sets a first predetermined value as a new difference corresponding to each pixel if the absolute value of the difference between at least one pixel element value is greater than or equal to the threshold TH and the difference is positive. The difference calculation unit 14 also sets a second predetermined value as a new difference corresponding to each pixel if the absolute value of the difference between at least one pixel element value is greater than or equal to the threshold TH and the difference is negative. The difference calculation unit 14 also sets a third predetermined value as a new difference corresponding to each pixel if the absolute value of the differences between N pixel element values ​​is less than the threshold TH.

[0076] The first predetermined value, the second predetermined value, and the third predetermined value are all different from each other. As a result, the difference data 130A is a collection of newly obtained differences (difference values) for each pixel. The storage unit 50 stores the difference data 130A. For example, as shown in Figure 5(c), the difference data 130A is composed of difference data 1120A having the first predetermined value, difference data 1220A having the second predetermined value, and difference data 1000A having the third predetermined value.

[0077] As described above with reference to Figure 5, according to this embodiment, instead of directly analyzing the three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522) to calculate the difference, the difference between two-dimensional images (first two-dimensional image 110 and second two-dimensional image 120) is calculated. Therefore, the processing load on the server 1 when calculating the difference can be reduced.

[0078] As an example, if the entire area of ​​the three-dimensional data (point cloud data) constituting the three-dimensional model is analyzed using an octree algorithm and the difference is calculated, the processing load on the computer is large. In contrast, in this embodiment, by calculating the difference between two-dimensional images (first two-dimensional image 110 and second two-dimensional image 120), the processing load on the server 1 is reduced, while indirectly calculating the difference between the three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522).

[0079] In particular, in this embodiment, as shown in Figure 3, the preprocessing for calculating the difference consists of generating a first two-dimensional image 110 that represents the first three-dimensional model 521 on a two-dimensional plane using a first viewpoint 61, and generating a second two-dimensional image 120 that represents the second three-dimensional model 522 on a two-dimensional plane using a second viewpoint 81, which is the same as the first viewpoint 61. By performing this simple preprocessing, the first two-dimensional image 110 and the second two-dimensional image 120 to be differed can be easily calculated.

[0080] Furthermore, in this embodiment, the image adjustment unit 13 in Figure 2 may perform an adjustment process on at least one of the first two-dimensional image 110 and the second two-dimensional image 120. In this case, the difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120 after the adjustment process has been performed.

[0081] The adjustment process includes at least one of the first adjustment process and the second adjustment process.

[0082] The first adjustment process is a process of adjusting the color tone of at least one of the first two-dimensional image 110 and the second two-dimensional image 120. The color tone is the brightness and / or contrast of the image. The image adjustment unit 13, for example, brings the color tone of one of the first two-dimensional image 110 and the second two-dimensional image 120 closer to the color tone of the other image, or matches it to the color tone of the other image.

[0083] For example, if the first time zone T1 is noon and the second time zone T2 is evening, the difference may be large across all pixels. In this case, even if the object has not changed between the first time zone T1 and the second time zone T2, the large difference may lead to the misinterpretation that the object has changed. Therefore, by adjusting or matching the color tones of the first two-dimensional image 110 and the second two-dimensional image 120 before calculating the difference, changes in the object can be detected with greater accuracy.

[0084] The second adjustment process is a process of excluding at least one pixel element from the N pixel elements that make up each pixel of the first two-dimensional image 110 and the second two-dimensional image 120 from the difference target. For example, the image adjustment unit 13 excludes from the difference calculation unit 14 a pixel element that depends on the surrounding environment of the object, among the N pixel elements that make up each pixel.

[0085] For example, in the evening, the B value may be larger than the R value and G value compared to noon. In this case, even if the object has not changed between the first time T1 and the second time T2, the large difference in the B value may lead to the object being judged as having changed. Therefore, for example, if the first time T1 is noon and the second time T2 is evening, the change in the object can be detected with high accuracy by excluding the B element from the R element, G element, and B element of each pixel from the difference calculation.

[0086] As described above, by providing the image adjustment unit 13, even if the surrounding environment of the object at the first time T1 when the object corresponding to the first three-dimensional model 521 was measured is different from the surrounding environment of the object at the second time T2 when the object corresponding to the second three-dimensional model 522 was measured, the difference between the first two-dimensional image 110 and the second two-dimensional image 120 can be accurately detected.

[0087] Note that the first and second adjustment processes are merely examples, and any adjustment process may be performed on the first two-dimensional image 110 and the second two-dimensional image 120.

[0088] Furthermore, in this embodiment, the image enhancement unit 15 in Figure 2 may enhance the areas in the two-dimensional image or three-dimensional model where the first two-dimensional image 110 and the second two-dimensional image 120 differ, based on the difference between the two-dimensional image 110 and the second two-dimensional image 120. In this case, when a user views the two-dimensional image or three-dimensional model, they can easily identify the differing areas. For example, the image enhancement unit 15 adds a specific color to the areas in the two-dimensional image or three-dimensional model where the first two-dimensional image 110 and the second two-dimensional image 120 differ, based on the difference between the two-dimensional image 110 and the second two-dimensional image 120.

[0089] Figure 6(a) shows an example of a two-dimensional image 140 that includes regions 1221 and 1121 that differ between the first two-dimensional image 110 and the second two-dimensional image 120. As shown in Figure 6(a), the image enhancement unit 15 enhances regions 1121 and 1221 in the two-dimensional image 140 for regions 141 that do not differ between the first two-dimensional image 110 and the second two-dimensional image 120, based on the difference data 130 or difference data 130A (Figure 5(c)). Region 1121 is identified based on the difference data 1120 or difference data 1120A (Figure 5(c)). Region 1221 is identified based on the difference data 1220 or difference data 1220A.

[0090] For example, region 1121 is highlighted by adding a first specific color 1122 (hatching with diagonal lines extending to the upper left). For example, region 1221 is highlighted by adding a second specific color 1222 (hatching with diagonal lines extending to the upper right).

[0091] The first specific color 1122 and the second specific color 1222 are different. Therefore, the user can easily distinguish between the region 1121 that exists in the first two-dimensional image 110 but not in the second two-dimensional image 120, and the region 1221 that does not exist in the first two-dimensional image 110 but does exist in the second two-dimensional image 120.

[0092] As an example, the image enhancement unit 15 uses the first two-dimensional image 110 or the second two-dimensional image 120 as a base image and generates the two-dimensional image 140 in Figure 6(a) based on the difference data 130 or difference data 130A in Figure 5(c). In this case, for example, the image enhancement unit 15 adds a first specific color 1122 to the regions identified by the difference data 1120 and 1120A in the first two-dimensional image 110 (Figure 5(a)) or the second two-dimensional image 120 (Figure 5(b)), and adds a second specific color 1222 to the regions identified by the difference data 1220 and 1220A. As a result, in the two-dimensional image 140, the regions 1121 and 1221 that differ between the first two-dimensional image 110 and the second two-dimensional image 120 are enhanced.

[0093] The memory unit 50 in Figure 2 stores the two-dimensional image 140. The display control unit 16 then displays the two-dimensional image 140 on the terminal 2 via the communication unit 40. As a result, the user of the terminal 2 can easily recognize the regions 1121 and 1221 where the first two-dimensional image 110 and the second two-dimensional image 120 differ.

[0094] Figure 6(b) shows an example of a three-dimensional model 140A that includes regions 1221A and 1121A where the first two-dimensional image 110 and the second two-dimensional image 120 differ. As shown in Figure 6(b), the three-dimensional model 140A is placed in a virtual space VS where a three-dimensional coordinate system CS is set. Based on the difference data 130 or difference data 130A (Figure 5(c)), the image enhancement unit 15 enhances regions 1121A and 1221A in the three-dimensional model 140A where the first two-dimensional image 110 and the second two-dimensional image 120 do not differ. Region 1121A is identified based on the difference data 1120 or difference data 1120A (Figure 5(c)) and is enhanced by adding a first specific color 1122. Region 1221A is identified based on differential data 1220 or differential data 1220A and is highlighted by the addition of a second specific color 1222.

[0095] As an example, the image enhancement unit 15 uses the first three-dimensional model 521 corresponding to the first two-dimensional image 110 or the second three-dimensional model 522 corresponding to the second two-dimensional image 120 as a base model, and generates the three-dimensional model 140A in Figure 6(b) based on the difference data 130 or difference data 130A in Figure 5(c). In this case, the first three-dimensional model 521 and the second three-dimensional model 522 are pre-assigned surfaces, and material information is attached to these surfaces.

[0096] For example, the image enhancement unit 15 adds a first specific color 1122 to the regions identified by the difference data 1120 and 1120A in the first three-dimensional model 521 or the second three-dimensional model 522, instead of material information. Furthermore, the image enhancement unit 15 adds a second specific color 1222 to the regions identified by the difference data 1220 and 1220A in the first three-dimensional model 521 or the second three-dimensional model 522, instead of material information. As a result, in the three-dimensional model 140A, the regions 1121A and 1221A where the first two-dimensional image 110 and the second two-dimensional image 120 differ are enhanced.

[0097] The memory unit 50 in Figure 2 stores the three-dimensional model 140A. The display control unit 16 then displays the three-dimensional model 140A on the terminal 2 via the communication unit 40. As a result, the user of the terminal 2 can easily recognize the regions 1121A and 1221A on the three-dimensional model 140A where the first two-dimensional image 110 and the second two-dimensional image 120 differ.

[0098] Here, the image generation unit 12 will be described with reference to Figures 2 and 3. The image generation unit 12 may set up multiple different first viewpoints 61 in the first virtual space VS1 (Figure 3(a)). The image generation unit 12 may then generate a first two-dimensional image 110 for each of the multiple different first viewpoints 61. Furthermore, the image generation unit 12 may set up multiple different second viewpoints 81 in the second virtual space VS2 (Figure 3(b)). The image generation unit 12 may then generate a second two-dimensional image 120 for each of the multiple different second viewpoints 81. Furthermore, the difference calculation unit 14 may calculate multiple differences corresponding to the multiple first viewpoints 61 and the multiple second viewpoints 81. Hereinafter, a first viewpoint 61 and a second viewpoint 81 identical to the first viewpoint 61 will be referred to as a "viewpoint pair". In this case, a first two-dimensional image 110 and a second two-dimensional image 120 are generated for each different viewpoint pair. Therefore, for each different viewpoint pair, a difference is generated between the first two-dimensional image 110 and the second two-dimensional image 120. In other words, multiple differences are generated for each of the multiple viewpoint pairs. Thus, the difference between the object at the first time T1 and the object at the second time T2 can be detected with high accuracy across the entire area of ​​the object.

[0099] For example, in the first virtual space VS1 shown in Figure 3(a), the image generation unit 12 sets multiple first viewpoints 61 at multiple points selected from vertices a1 to a8, midpoints b1 to b12, and centers c1 to c6. Then, in the second virtual space VS2 shown in Figure 3(b), the image generation unit 12 sets multiple second viewpoints 81 that are the same as the multiple first viewpoints 61.

[0100] Next, an image analysis method according to this embodiment will be described with reference to Figures 2 and 7. Figure 7 is a flowchart of the image analysis method. The image analysis method is executed by the server 1. As shown in Figure 7, the image analysis method includes steps S1 to S12. The computer program stored in the storage unit 50 causes the arithmetic unit 10 to execute steps S1 to S12. In other words, the computer program product realizes steps S1 to S12 when the computer program is executed by the arithmetic unit 10. The arithmetic unit 10 corresponds to an example of a "computer" in this disclosure.

[0101] As shown in Figures 2 and 7, first, in step S1, the model generation unit 11 acquires a first three-dimensional model 521. Specifically, the model generation unit 11 generates a first three-dimensional model 521 representing the object at first time T1 based on multiple two-dimensional images generated by imaging the object from multiple different imaging positions at first time T1. Alternatively, the model generation unit 11 sets three-dimensional data 513 based on the measurement results of the object at first time T1 into the first three-dimensional model 521. Then, the model generation unit 11 sets a first rectangular parallelepiped 71 surrounding the first three-dimensional model 521.

[0102] Next, in step S2, the image generation unit 12 generates a first two-dimensional image 110 in which the first three-dimensional model 521 is represented on a two-dimensional plane by a first viewpoint 61 in the first virtual space VS1. The first viewpoint 61 is set on the first rectangular parallelepiped 71.

[0103] Next, in step S3, the model generation unit 11 acquires a second three-dimensional model 522. Specifically, the model generation unit 11 generates a second three-dimensional model 522 representing the object in the second time T2 based on multiple two-dimensional images generated by imaging the object from multiple different imaging positions in the second time T2. Alternatively, the model generation unit 11 sets three-dimensional data 513 based on the measurement results of the object in the second time T2 into the second three-dimensional model 522. Then, the model generation unit 11 sets a second rectangular parallelepiped 72 surrounding the second three-dimensional model 522. In this disclosure, setting the second rectangular parallelepiped 72 is not mandatory.

[0104] Next, in step S4, the image generation unit 12 generates a second two-dimensional image 120 in which the second three-dimensional model 522 is represented on a two-dimensional plane by the second viewpoint 81 of the second virtual space VS2. In the three-dimensional coordinate system CS set in the first virtual space VS1 and the second virtual space VS2, the position and line of sight direction 82 of the second viewpoint 81 are the same as the position and line of sight direction 82 of the first viewpoint 61, respectively.

[0105] Next, in step S5, the image adjustment unit 13 performs an adjustment process on at least one of the first two-dimensional image 110 and the second two-dimensional image 120.

[0106] Next, in step S6, the difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120 after the adjustment process is performed, and generates difference data 130.

[0107] Next, in step S7, the image enhancement unit 15 enhances the difference region between the first two-dimensional image 110 and the second two-dimensional image 120 in the two-dimensional image 140 or the three-dimensional model 140A based on the difference indicated by the difference data 130.

[0108] Next, in step S8, the image generation unit 12 determines whether or not it has generated difference data 130 indicating the difference for all viewpoint pairs (first viewpoint 61 and second viewpoint 81).

[0109] If a negative result is obtained in step S8 (NO), the process proceeds to step S9.

[0110] Next, in step S9, the image generation unit 12 changes the viewpoint pair (first viewpoint 61 and second viewpoint 81).

[0111] Next, in step S10, the image generation unit 12 generates a first two-dimensional image 110 in which the first three-dimensional model 521 is represented on a two-dimensional plane with the modified first viewpoint 61.

[0112] Next, in step S11, the image generation unit 12 generates a second two-dimensional image 120, which represents the second three-dimensional model 522 on a two-dimensional plane using the modified second viewpoint 81. Then, the process proceeds to step S5. Steps S5 to S11 are repeated until difference data 130 is calculated for all viewpoint pairs (first viewpoint 61 and second viewpoint 81).

[0113] On the other hand, if a positive determination is made in step S8 (YES), the process proceeds to step S12.

[0114] Next, in step S12, the display control unit 16, in response to a request from the user's terminal 2, displays a two-dimensional image 140 or a three-dimensional model 140A on the terminal 2 via the communication unit 40, in which the difference region between the first two-dimensional image 110 and the second two-dimensional image 120 is highlighted. The image analysis method then ends.

[0115] As described above with reference to Figure 7, the image analysis method according to this embodiment calculates the difference between two-dimensional images (first two-dimensional image 110 and second two-dimensional image 120) rather than directly analyzing three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522). Therefore, the processing load on the server 1 when calculating the difference can be reduced.

[0116] Preferred embodiments and modifications of the present disclosure have been described in detail above with reference to the attached drawings, but the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations can be conceived within the scope of the technical idea set forth in the claims, and these too are understood to fall within the technical scope of the present disclosure.

[0117] The apparatus or system described herein may be implemented as a single apparatus, or it may be implemented by a plurality of apparatuses (e.g., a cloud server) that are partially or completely connected by a network. For example, some or all of the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may be implemented by the same computer or server. For example, the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may each be implemented by separate computers or servers. Alternatively, for example, the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may be implemented by a terminal 2 or a three-dimensional measuring device 7. For example, video data 511, still image dataset 512, three-dimensional data 513, first three-dimensional model 521, second three-dimensional model 522, first two-dimensional image 110, second two-dimensional image 120, difference data 130, 130A, two-dimensional image 140, and three-dimensional model 140A may each be stored in separate storage devices or servers.

[0118] The series of processes performed by the apparatus described herein may be implemented using software, hardware, or a combination of software and hardware. Computer programs for implementing each function of the arithmetic unit 10 according to this embodiment can be created and implemented on a PC or the like. Furthermore, a computer-readable recording medium containing such a computer program can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, and flash memory. Alternatively, the computer program may be distributed without using a recording medium, for example, via a network.

[0119] Furthermore, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0120] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0121] Furthermore, the following configurations also fall within the technical scope of this disclosure.

[0122] (Item 1) An image analysis device comprising: an image generation unit that generates a first two-dimensional image and a second two-dimensional image; and a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image, wherein the image generation unit generates a first two-dimensional image in which a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time is represented on a two-dimensional plane from a first viewpoint in a first virtual space; and generates a second two-dimensional image in which a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time is represented on a two-dimensional plane from a second viewpoint in a second virtual space; the same coordinate system is set for the first virtual space and the second virtual space; and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0123] (Item 2) The first viewpoint is set on a rectangular parallelepiped surrounding the first three-dimensional model, and is the image analysis device described in Item 1.

[0124] (Item 3) The image analysis device described in Item 2, wherein the first viewpoint is set at a vertex, midpoint of an edge, or center of a face of the rectangular parallelepiped.

[0125] (Item 4) The image analysis device described in Item 2, wherein the first viewpoint is set at a vertex of the rectangular parallelepiped and faces the direction of the diagonal line extending from the vertex.

[0126] (Item 5) An image analysis device according to any one of Items 1 to 4, wherein the image generation unit sets a plurality of different first viewpoints in the first virtual space and generates a first two-dimensional image for each of the plurality of different first viewpoints, the image generation unit sets a plurality of different second viewpoints in the second virtual space and generates a second two-dimensional image for each of the plurality of different second viewpoints, and the difference calculation unit calculates a plurality of differences corresponding to the plurality of first viewpoints and the plurality of second viewpoints.

[0127] (Item 6) An image analysis device according to any one of items 1 to 5, further comprising an image adjustment unit that performs an adjustment process on at least one of the first two-dimensional image and the second two-dimensional image, wherein the adjustment process includes at least one of a first adjustment process and a second adjustment process, the first adjustment process is a process that adjusts the color tone of the at least one of the images, the second adjustment process is a process that excludes at least one pixel element from among a plurality of pixel elements constituting each pixel of the first two-dimensional image and the second two-dimensional image from the difference target, and the difference calculation unit calculates the difference between the first two-dimensional image and the second two-dimensional image after the adjustment process is performed.

[0128] (Item 7) An image analysis device according to any one of items 1 to 6, further comprising an image enhancement unit that enhances the regions in the two-dimensional image or three-dimensional model that differ from the first two-dimensional image based on the difference.

[0129] (Item 8) An image analysis method comprising the steps of: generating a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space; and calculating the difference between the first two-dimensional image and the second two-dimensional image, wherein the same coordinate system is set for the first virtual space and the second virtual space, and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0130] (Item 9) A computer program that causes a computer to perform the following steps: generate a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; generate a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space; and calculate the difference between the first two-dimensional image and the second two-dimensional image, wherein the same three-dimensional coordinate system is set in the first virtual space and the second virtual space, and in the three-dimensional coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0131] This disclosure provides an image analysis device, an image analysis method, and a computer program, and has industrial applicability.

[0132] 1 Server (image analysis device), 11 Model generation unit, 12 Image generation unit, 13 Image adjustment unit, 14 Difference calculation unit, 15 Image enhancement unit, 16 Display control unit

Claims

1. An image analysis device comprising: an image generation unit that generates a first two-dimensional image and a second two-dimensional image; and a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image, wherein the image generation unit generates a first two-dimensional image in which a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time is represented on a two-dimensional plane from a first viewpoint in a first virtual space; and generates a second two-dimensional image in which a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time is represented on a two-dimensional plane from a second viewpoint in a second virtual space, wherein the same coordinate system is set for the first virtual space and the second virtual space, and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

2. The image analysis apparatus according to claim 1, wherein the first viewpoint is set on a rectangular parallelepiped surrounding the first three-dimensional model.

3. The image analysis device according to claim 2, wherein the first viewpoint is set at a vertex, midpoint of an edge, or center of a face of the rectangular parallelepiped.

4. The image analysis apparatus according to claim 2, wherein the first viewpoint is set at a vertex of the rectangular parallelepiped and faces the direction of the diagonal line extending from the vertex.

5. The image analysis apparatus according to claim 1 or 2, wherein the image generation unit sets a plurality of different first viewpoints in the first virtual space and generates a first two-dimensional image for each of the plurality of different first viewpoints, the image generation unit sets a plurality of different second viewpoints in the second virtual space and generates a second two-dimensional image for each of the plurality of different second viewpoints, and the difference calculation unit calculates a plurality of differences corresponding to the plurality of first viewpoints and the plurality of second viewpoints.

6. An image analysis device according to claim 1 or 2, further comprising an image adjustment unit that performs an adjustment process on at least one of the first two-dimensional image and the second two-dimensional image, wherein the adjustment process includes at least one of a first adjustment process and a second adjustment process, the first adjustment process is a process to adjust the color tone of the at least one of the images, the second adjustment process is a process to exclude at least one pixel element from among a plurality of pixel elements constituting each pixel of the first two-dimensional image and the second two-dimensional image from the difference target, and the difference calculation unit calculates the difference between the first two-dimensional image and the second two-dimensional image after the adjustment process is performed.

7. The image analysis apparatus according to claim 1 or claim 2, further comprising an image enhancement unit that enhances the regions in the two-dimensional image or three-dimensional model that differ from the first two-dimensional image based on the difference.

8. An image analysis method comprising the steps of: generating a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space; and calculating the difference between the first two-dimensional image and the second two-dimensional image, wherein the same coordinate system is set for the first virtual space and the second virtual space, and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

9. A computer program that causes a computer to perform the following steps: generate a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; generate a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space; and calculate the difference between the first two-dimensional image and the second two-dimensional image, wherein the same three-dimensional coordinate system is set in the first virtual space and the second virtual space, and in the three-dimensional coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

Images