Image analysis device, image analysis method, and computer program

Abstract

The image analysis device includes an image generation unit that generates a first two-dimensional image and a second two-dimensional image, and a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image. The image generation unit generates a first two-dimensional image by representing a first three-dimensional model generated based on measurement results of the shape of an object in real space at a first time on a two-dimensional plane from a first viewpoint in a first virtual space. The image generation unit generates a second two-dimensional image by representing a second three-dimensional model generated based on measurement results of the shape of the object in real space at a second time on a two-dimensional plane from a second viewpoint in a second virtual space. The same coordinate system is set in the first virtual space and the second virtual space. In the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

Description

[Technical Field]

[0001] The present disclosure relates to an image analysis device, an image analysis method, and a computer program. [Background technology]

[0002] The differential detection device described in Patent Document 1 photographs a structure from multiple directions to acquire multiple images during a first inspection, and generates first three-dimensional point cloud data from the multiple images.The differential detection device then photographs the structure from multiple directions again during a second inspection, and generates second three-dimensional point cloud data from the multiple images.The first point cloud data and second point cloud data represent the same location on the structure.

[0003] The differential detection device compares color information indicating the colors of points included in the first point cloud data with color information indicating the colors of points included in the second point cloud data, and when the difference between the two is equal to or greater than a threshold, the differential detection device changes the color of the second point cloud data where the difference is equal to or greater than the threshold. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-181056 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the differential detection device described in Patent Document 1 calculates the difference by analyzing the entire area of ​​the three-dimensional data (first point cloud data and second point cloud data), which results in an excessive processing load on the differential detection device.

[0006] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and its purpose is to provide an image analysis device, an image analysis method, and a computer program that can reduce the processing load when calculating the difference. [Means for solving the problem]

[0007] According to the present disclosure, there is provided an image analysis device comprising an image generation unit that generates a first two-dimensional image and a second two-dimensional image, and a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image, wherein the image generation unit generates the first two-dimensional image, which represents a first three-dimensional model generated based on measurement results of the shape of an object in real space at a first time, on a two-dimensional plane from a first viewpoint in a first virtual space, and generates the second two-dimensional image, which represents a second three-dimensional model generated based on measurement results of the shape of the object in real space at a second time, on a two-dimensional plane from a second viewpoint in a second virtual space, wherein the same coordinate system is set in the first virtual space and the second virtual space, and in the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

[0008] The present disclosure also provides an image analysis method including the steps of: generating a first two-dimensional image in a first virtual space, the first two-dimensional image representing a first three-dimensional model generated based on measurement results of the shape of an object in real space at a first time, on a two-dimensional plane from a first viewpoint; generating a second two-dimensional image in a second virtual space, the second three-dimensional model generated based on measurement results of the shape of the object in the real space at a second time, on a two-dimensional plane from a second viewpoint; and calculating a difference between the first two-dimensional image and the second two-dimensional image, wherein the same coordinate system is set in the first virtual space and the second virtual space, and in the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

[0009] Furthermore, according to the present disclosure, there is provided a computer program that causes a computer to execute the steps of: generating a first two-dimensional image that represents a first three-dimensional model generated based on measurement results of the shape of an object in real space at a first time, on a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image that represents a second three-dimensional model generated based on measurement results of the shape of the object in the real space at a second time, on a two-dimensional plane from a second viewpoint in a second virtual space; and calculating the difference between the first two-dimensional image and the second two-dimensional image, wherein the same three-dimensional coordinate system is set in the first virtual space and the second virtual space, and in the three-dimensional coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to provide an image analysis device, an image analysis method, and a computer program that can reduce the processing load when calculating a difference. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of an image analysis system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of a server according to the embodiment. [Figure 3] 1A is a perspective view schematically showing a first viewpoint, a first three-dimensional model, and a first rectangular parallelepiped in a first virtual space according to the embodiment, and FIG. 1B is a perspective view schematically showing a second viewpoint, a second three-dimensional model, and a second rectangular parallelepiped in a second virtual space according to the embodiment. [Figure 4] 1A is a side view schematically showing a first viewpoint, a first three-dimensional model, and a first rectangular parallelepiped in a first virtual space according to the embodiment, and FIG. 1B is a side view schematically showing a second viewpoint, a second three-dimensional model, and a second rectangular parallelepiped in a second virtual space according to the embodiment. [Figure 5]1A is a diagram showing an example of a first two-dimensional image according to the embodiment, FIG. 1B is a diagram showing an example of a second two-dimensional image according to the embodiment, and FIG. 1C is a diagram showing the concept of difference data according to the embodiment. [Figure 6] 1A is a diagram showing an example of a two-dimensional image including a region where the first and second two-dimensional images according to the embodiment differ, and FIG. 1B is a diagram showing an example of a three-dimensional model including a region where the first and second two-dimensional images according to the embodiment differ. [Figure 7] 10 is a flowchart showing an image analysis method according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. In this embodiment, the rectangular parallelepiped includes a cube.

[0013] Fig. 1 is a diagram showing an example of the configuration of an image analysis system SYS according to an embodiment of the present invention. As shown in Fig. 1, the image analysis system SYS includes a server 1. The server 1 corresponds to an example of the "image analysis device" of the present disclosure.

[0014] To detect differences between two three-dimensional models generated by measuring the same object at different times, the server 1 calculates the difference between two two-dimensional images that represent the two three-dimensional models on a two-dimensional plane from the same viewpoint. As described above, according to this embodiment, the difference between the two-dimensional images is calculated, which reduces the processing load on the server 1 when calculating the difference compared to when calculating the difference by analyzing the entire area of ​​the three-dimensional data that constitutes the three-dimensional model. This point will be described in detail later.

[0015] The image analysis system SYS also includes at least one terminal 2, at least one mobile device 3, at least one imaging device 5, or at least one three-dimensional measuring device .

[0016] The server 1, the terminal 2, the mobile device 3, the imaging device 5, and the three-dimensional measuring device 7 are connected to a network NW. The network NW includes, for example, the Internet, a closed network, a public telephone network, a LAN (Local Area Network), and a short-range wireless network.

[0017] The terminal 2 is, for example, a personal computer (for example, a notebook computer, a desktop computer, or a tablet).

[0018] The mobile device 3 is, for example, an unmanned mobile device or a manned mobile device. The unmanned mobile device is, for example, an unmanned aerial vehicle such as a drone, an unmanned ground vehicle, an unmanned underwater vehicle, or an unmanned surface vessel. The unmanned ground vehicle is, for example, an unmanned ground vehicle modeled after a living organism (for example, a snake-shaped unmanned ground vehicle). The manned mobile device is, for example, an aircraft, an automobile, a ship, or a submarine. The mobile device 3 includes a camera 4.

[0019] The imaging device 5 includes a camera 6. The imaging device 5 is, for example, a mobile terminal such as a smartphone. The imaging device 5 may be, for example, the camera 6 itself.

[0020] The cameras 4 and 6 capture images of an object and generate video data representing a video including an image of the object. A video is a collection of successive two-dimensional images. The cameras 4 and 6 may also generate multiple still image data representing multiple still images including an image of the object. The still images are two-dimensional images.

[0021] Hereinafter, the video data will be referred to as "video data 511" (FIG. 2), and the plurality of still image data will be referred to as "still image data set 512" (FIG. 2).

[0022] Hereinafter, when there is no need to distinguish between cameras 4 and 6, cameras 4 and 6 will be collectively referred to as "camera CM."

[0023] The three-dimensional measuring device 7 measures the shape of an object and generates three-dimensional data (hereinafter referred to as "three-dimensional data 513") representing the shape of the object. The three-dimensional data 513 is typically point cloud data. A three-dimensional model of the object is formed by the three-dimensional data 513. The three-dimensional measuring device 7 may be a contact type or a non-contact type. The three-dimensional measuring device 7 may also use an active method or a passive method. Examples of active methods include an optical radar method (ToF (Time of Flight) method), an active stereo method, or an optical interferometry method. The optical radar method measures the shape of an object by irradiating the object with light and measuring the change in the time or phase until the reflected light returns to a detector. An example of the optical radar method is LiDAR (Light Detection and Ranging). The active stereo method measures the shape of an object by projecting a laser beam, a slit light, or a code pattern of light. The optical interferometry method measures the shape of an object by irradiating the object with light and utilizing the interference of the light. An example of a passive method is the lens focus method.

[0024] Furthermore, the objects captured by the camera CM and the objects measured by the three-dimensional measuring device 7 are not particularly limited as long as they can be captured by the camera CM or measured by the three-dimensional measuring device 7. Furthermore, for example, the size, shape, pattern, and color of the objects are also not particularly limited. For example, the objects are one or more movable or immovable property. The objects are, for example, one or more objects. Typically, the objects are one or more stationary objects. Stationary objects are, for example, man-made or natural objects. Man-made objects are, for example, structures, machines, electronic devices, or copyrighted works. Structures are, for example, buildings or infrastructure facilities. Buildings are, for example, buildings or houses. Infrastructure facilities are facilities for establishing social infrastructure. For example, infrastructure facilities are roads, bridges, road traffic facilities, power generation facilities, power distribution facilities, water treatment facilities, or gas distribution facilities. Machines are, for example, automobiles, work vehicles, trains, aircraft, ships, submarines, or robots. The natural object is, for example, a tree, a forest, the ground, a cliff, a coast, or a river.

[0025] The terminal 2 acquires video data 511 or still image data set 512 generated by the camera CM, or three-dimensional data 513 generated by the three-dimensional measurement device 7. For example, the terminal 2 receives the video data 511 or still image data set 512 transmitted from the camera CM, or the three-dimensional data 513 transmitted from the three-dimensional measurement device 7, via the network NW. The terminal 2 transmits the video data 511, the still image data set 512, and the three-dimensional data 513 to the server 1 via the network NW. Note that the mobile device 3 and the imaging device 5 may transmit the video data 511 or the still image data set 512 to the server 1 via the network NW. Alternatively, the three-dimensional measurement device 7 may transmit the three-dimensional data 513 to the server 1 via the network NW.

[0026] Fig. 2 is a block diagram showing an example configuration of the server 1 in Fig. 1. As shown in Fig. 2, the server 1 includes a calculation unit 10, a communication unit 40, and a storage unit 50. The server 1 may also include an input unit 20 and a display unit 30.

[0027] The input unit 20 is an input device for inputting various pieces of information to the calculation unit 10. For example, the input unit 20 is a keyboard and pointing device, or a touch panel.

[0028] The display unit 30 displays various types of information and is, for example, a liquid crystal display or an organic electroluminescence display.

[0029] The communication unit 40 is connected to the network NW. The communication unit 40 communicates with external devices connected to the network NW. The external devices are, for example, the terminal 2, the mobile device 3, the imaging device 5, and the three-dimensional measuring device 7. The communication unit 40 is a communication device that communicates according to a predetermined communication protocol, and includes, for example, a network interface controller. The predetermined communication protocol is, for example, a protocol compliant with Ethernet (registered trademark) and the Internet Protocol Suite.

[0030] The communication unit 40 receives video data 511 or still image data set 512 from the terminal 2, the mobile device 3, and the imaging device 5 via the network NW. The communication unit 40 also receives three-dimensional data 513 from the terminal 2 and the three-dimensional measurement device 7 via the network NW.

[0031] The storage unit 50 includes a storage device and stores data and computer programs. The storage unit 50 includes a main storage device such as a semiconductor memory, and an auxiliary storage device such as a semiconductor memory and a hard disk drive. The storage unit 50 may also include removable media such as an optical disk. The storage unit 50 may be, for example, a non-transitory computer-readable storage medium.

[0032] The storage unit 50 stores video data 511, a still image data set 512, and three-dimensional data 513. The video data 511, the still image data set 512, and the three-dimensional data 513 are associated with attribute information (hereinafter, "attribute information AT"). The attribute information AT includes, for example, user information, data acquisition conditions, and object information. The user information includes, for example, identification information of the user of the terminal 2, the mobile device 3, the imaging device 5, or the three-dimensional measurement device 7. The data acquisition conditions include, for example, the imaging time or the measurement time. The imaging time and the measurement time are indicated by one or more of the year, month, date, and time. The data acquisition conditions may include information on the imaging location or the measurement location. The imaging location and the measurement location are indicated by, for example, the position coordinates of the camera CM and the three-dimensional measurement device 7 acquired by a global positioning system (GPS) or a global navigation satellite system (GNSS), respectively. The object information includes, for example, identification information of the object. The object information may include coordinates of ground control points (GCPs).

[0033] The storage unit 50 also stores a first three-dimensional model 521, a second three-dimensional model 522, a first two-dimensional image 110, a second two-dimensional image 120, differential data 130 (130A), a two-dimensional image 140, and a three-dimensional model 140A. These data are stored when they are generated. Details of these data will be described later.

[0034] The calculation unit 10 executes various calculations and includes processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

[0035] Specifically, the calculation unit 10 includes an image generation unit 12 and a difference calculation unit 14. The calculation unit 10 may also include a model generation unit 11, an image adjustment unit 13, an image enhancement unit 15, and a display control unit 16. For example, the calculation unit 10 functions as the model generation unit 11, the image generation unit 12, the image adjustment unit 13, the difference calculation unit 14, the image enhancement unit 15, and the display control unit 16 by executing a computer program stored in the storage unit 50.

[0036] The model generation unit 11 acquires a plurality of two-dimensional images generated by capturing an object from a plurality of different imaging positions from the video data 511 or the still image dataset 512 in the storage unit 50. The model generation unit 11 then generates a three-dimensional model of the object based on the plurality of two-dimensional images. The three-dimensional model is placed in a virtual space. The three-dimensional model represents the three-dimensional shape of the object. The three-dimensional model is configured by point cloud data. The point cloud data is data representing a point cloud. A point cloud is a collection of a plurality of points. The point cloud data includes three-dimensional coordinates of each point. The point cloud data may further include one or more of color information (e.g., RGB values) of each point, normal vector information of each point, and reflection intensity information.

[0037] As an example, the model generation unit 11 generates a three-dimensional model by executing SfM (Structure from Motion) processing. SfM processing refers to processing that generates a three-dimensional model by utilizing the principle of triangulation based on multiple two-dimensional images generated by capturing images of an object having multiple feature points from multiple imaging positions. The SfM processing preferably includes bundle adjustment. Bundle adjustment refers to processing that minimizes reprojection error. In addition to the SfM processing, the model generation unit 11 may also execute MVS (Multi View Stereo) processing. MVS processing refers to processing that calculates depth and normals for each pixel of each two-dimensional image using multi-view stereo measurement, integrates them, and generates a dense point cloud of the object.

[0038] Furthermore, it is preferable that the model generation unit 11 assigns surfaces to the three-dimensional model based on the point cloud data. The process of assigning surfaces is, for example, a process of converting point cloud data constituting the three-dimensional model into mesh data. In this case, for example, the model generation unit 11 generates a plurality of polygonal (e.g., triangular) surfaces (e.g., polygons) generated by connecting points of the point cloud data, and represents the three-dimensional model using the plurality of polygonal surfaces. In this case, for example, the model generation unit 11 generates TIN (Triangulated Irregular Network) data based on the point cloud data, and represents the three-dimensional model using the TIN data. As a result, the three-dimensional model is represented by a collection of triangular surfaces. Note that the process of assigning surfaces may also be, for example, a process of converting point cloud data into surface data.

[0039] Furthermore, the model generation unit 11 may add material information based on the two-dimensional image to the surfaces of the three-dimensional model. The material information is information including the color and pattern of the object. For example, the model generation unit 11 may perform a process of mapping texture to each surface (e.g., each polygon) constituting the mesh data of the three-dimensional model.

[0040] As described above with reference to Figure 2, the three-dimensional model is generated based on the results of capturing an image of an object by the camera CM. Therefore, capturing an image of an object by the camera CM can be considered as measuring the shape of the object. Therefore, the time when the object is captured by the camera CM is the time when the object is measured by the camera CM.

[0041] The three-dimensional data 513 generated by the three-dimensional measuring device 7 is similar to the point cloud data constituting the three-dimensional model generated by the model generation unit 11. Similarly, surfaces may be added to the three-dimensional model represented by the three-dimensional data 513, and material information may be added to the surfaces.

[0042] The storage unit 50 stores the three-dimensional model generated by the model generation unit 11 as a first three-dimensional model 521. The storage unit 50 also stores another three-dimensional model generated by the model generation unit 11 as a second three-dimensional model 522. The three-dimensional data 513 generated by the three-dimensional measurement device 7 may be treated as the first three-dimensional model 521, and the other three-dimensional data 513 may be treated as the second three-dimensional model 522. These details will be described later.

[0043] Note that the methods for generating the three-dimensional data 513, the first three-dimensional model 521, and the second three-dimensional model 522 are exemplified by the method executed by the three-dimensional measurement device 7 and SfM processing. However, as long as the three-dimensional data or the three-dimensional model can be generated, the generation method is not particularly limited. For example, the three-dimensional data or the three-dimensional model may be generated by 3D Gaussian Splatting or Neural Radiance Field (NeRF). Furthermore, the three-dimensional data or the three-dimensional model is not limited to point cloud data, and the data format is not particularly limited.

[0044] Next, the image generation unit 12 will be described with reference to FIGS. 2 to 4. FIG. 3(a) is a perspective view schematically showing a first viewpoint 61, a first three-dimensional model 521, and a first rectangular parallelepiped 71 in a first virtual space VS1. FIG. 3(b) is a perspective view schematically showing a second viewpoint 81, a second three-dimensional model 522, and a second rectangular parallelepiped 72 in a second virtual space VS2. FIG. 4(a) is a side view schematically showing the first viewpoint 61, the first three-dimensional model 521, and the first rectangular parallelepiped 71 in the first virtual space VS1. FIG. 4(a) corresponds to FIG. 3(a). FIG. 4(b) is a side view schematically showing the second viewpoint 81, the second three-dimensional model 522, and the second rectangular parallelepiped 72 in a second virtual space VS2. FIG. 4(b) corresponds to FIG. 3(b). 3 and 4, a virtual reference plane 75 is shown for ease of understanding.

[0045] As shown in FIGS. 2 and 3(a), a first three-dimensional model 521 is placed in a first virtual space VS1. The first three-dimensional model 521 is a three-dimensional model generated based on the measurement results of the shape of an object at a first time (hereinafter referred to as "first time T1") in real space. The first time T1 is indicated by attribute information AT. For example, the first time T1 is indicated by one or more of the year, month, day, and time.

[0046] A three-dimensional coordinate system CS is set in the first virtual space VS1. The three-dimensional coordinate system CS is defined by an X-axis, a Y-axis, and a Z-axis, which are orthogonal to each other. The three-dimensional coordinate system CS may be a coordinate system with an origin O at a predetermined position in the first virtual space VS1, or may be a coordinate system to which geospatial coordinates (ground coordinates) or actual size information is assigned. As an example, for the sake of convenience in explanation, in FIGS. 3 and 4, the origin O of the three-dimensional coordinate system CS is set on a virtual reference plane 75. The three-dimensional coordinate system CS corresponds to an example of a "coordinate system" in the present disclosure.

[0047] The image generation unit 12 generates a first two-dimensional image 110 that represents the first three-dimensional model 521 on a two-dimensional plane from a first viewpoint 61 in a first virtual space VS1. The first viewpoint 61 faces a line of sight 62 at a position p1 in the first virtual space VS1. In other words, the image generation unit 12 generates the first two-dimensional image 110 by capturing the first three-dimensional model 521 from the first viewpoint 61. In further other words, the image generation unit 12 generates the first two-dimensional image 110 by pseudo-capturing the first three-dimensional model 521 with a first virtual camera 63 having the first viewpoint 61. The storage unit 50 stores the first two-dimensional image 110.

[0048] In detail, the model generation unit 11 sets a first rectangular parallelepiped 71 for the first three-dimensional model 521 in the first virtual space VS1. The first rectangular parallelepiped 71 surrounds the first three-dimensional model 521. The first rectangular parallelepiped 71 has a shape and size corresponding to the shape and size of the first three-dimensional model 521. The first rectangular parallelepiped 71 is a rectangular parallelepiped that defines the display range when the first three-dimensional model 521 is displayed in a viewer. Hereinafter, a rectangular parallelepiped that defines the display range when the three-dimensional model is displayed in a viewer may be referred to as a "defining rectangular parallelepiped." The viewer is software or equipment for viewing images. The defining rectangular parallelepiped is, for example, a bounding box. The first rectangular parallelepiped 71 corresponds to an example of a "rectangular parallelepiped" in the present disclosure.

[0049] The first viewpoint 61 is set on the first rectangular parallelepiped 71. Therefore, according to this embodiment, the first viewpoint 61 can be set more easily than when the first viewpoint 61 is set at an arbitrary point in the first virtual space VS1. In particular, the first rectangular parallelepiped 71 is always set when the first three-dimensional model 521 is displayed in the viewer, so that it is possible to prevent additional processing from occurring just to determine the first viewpoint 61.

[0050] Preferably, the first viewpoint 61 is set at any one of the eight vertices a1, a2, a3, a4, a5, a6, a7, and a8 of the first rectangular parallelepiped 71, any one of the midpoints b1, b2, b3, b4, b5, b6, b7, b8, b9, b10, b11, and b12 of the twelve sides of the first rectangular parallelepiped 71, or any one of the centers c1, c2, c3, c4, c5, and c6 of the six faces of the first rectangular parallelepiped 71. Therefore, according to this preferred example, the first viewpoint 61 can be set more easily.

[0051] For example, when first viewpoint 61 is set at one of vertices a1 to a8, line of sight 62 of first viewpoint 61 faces in the direction of a diagonal of first rectangular parallelepiped 71. In other words, line of sight 62 faces the vertex diagonally opposite to the vertex at which first viewpoint 61 is set.

[0052] When the first viewpoint 61 is set to one of the vertices a1 to a8, it is possible to generate a first two-dimensional image 110 of the first three-dimensional model 521 viewed from, for example, diagonally above or diagonally below. In particular, since the positions of the vertices a1 to a8 are uniquely determined, the process of calculating the positions of the vertices a1 to a8 can be omitted.

[0053] 3(a), the position p1 of the first viewpoint 61 is at the vertex a1. Specifically, the first viewpoint 61 is set at the vertex a1 of the first rectangular parallelepiped 71, and faces in the direction of a diagonal line 73 extending from the vertex a1. In other words, the line of sight 62 faces along the diagonal line 73 from the vertex a1 to the vertex a7.

[0054] For example, when first viewpoint 61 is set at one of midpoints b1 to b12, line of sight direction 62 of first viewpoint 61 points toward a midpoint diagonally opposite to the midpoint at which first viewpoint 61 is set, across first 3D model 521. In this case, for example, when first viewpoint 61 is set at midpoint b1, line of sight direction 62 points from midpoint b1 to midpoint b6.

[0055] When the first viewpoint 61 is set to one of the midpoints b1 to b12, it is possible to generate a first two-dimensional image 110 when the first three-dimensional model 521 is viewed obliquely from above, below, or from the side.

[0056] For example, when first viewpoint 61 is set at one of centers c1 to c6, line of sight direction 62 of first viewpoint 61 points toward a center opposite to the center at which first viewpoint 61 is set, in a direction perpendicular to a plane including the center at which first viewpoint 61 is set. In this case, for example, when first viewpoint 61 is set at center c1, line of sight direction 62 points from center c1 to center c3.

[0057] When the first viewpoint 61 is set to one of the centers c1 to c6, for example, a first two-dimensional image 110 of the first three-dimensional model 521 viewed from a direction perpendicular to the surface of the first rectangular parallelepiped 71 can be generated.

[0058] On the other hand, as shown in FIG. 3(b), the second three-dimensional model 522 is placed in a second virtual space VS2. The second three-dimensional model 522 is a three-dimensional model generated based on the measurement results of the shape of an object at a second time (hereinafter referred to as "second time T2") in real space. The second time T2 is indicated by attribute information AT. For example, the second time T2 is indicated by one or more of the year, month, day, and time. The second time T2 is different from the first time T1. For example, the second time T2 is a time that occurs earlier than the first time T1.

[0059] A three-dimensional coordinate system CS is set in the second virtual space VS2, that is, the same three-dimensional coordinate system CS is set in the second virtual space VS2 and the first virtual space VS1.

[0060] The image generation unit 12 generates a second two-dimensional image 120 that represents the second three-dimensional model 522 on a two-dimensional plane from a second viewpoint 81 in the second virtual space VS2. The second viewpoint 81 faces a line of sight 82 at a position p2 in the second virtual space VS2. In other words, the image generation unit 12 generates the second two-dimensional image 120 by capturing the second three-dimensional model 522 from the second viewpoint 81. In further other words, the image generation unit 12 generates the second two-dimensional image 120 by pseudo-capturing the second three-dimensional model 522 with a second virtual camera 83 having the second viewpoint 81. The storage unit 50 stores the second two-dimensional image 120.

[0061] In particular, as shown in Figures 3(a) and 3(b), in the three-dimensional coordinate system CS, the position p2 and line of sight direction 82 of the second viewpoint 81 (second virtual camera 83) are the same as the position p1 and line of sight direction 62 of the first viewpoint 61 (first virtual camera 63), respectively.

[0062] 4(a) and 4(b), the three-dimensional coordinates (aX, aY, aZ) indicating the position p2 of the second viewpoint 81 are the same as the three-dimensional coordinates (aX, aY, aZ) indicating the position p1 of the first viewpoint 61. Furthermore, the line-of-sight directions 62 and 82 (the attitudes of the first virtual camera 63 and the second virtual camera 83) are expressed by the same rotation angle (θx, θy, θz) with respect to the reference direction (reference attitude). θx indicates the rotation angle around the X-axis, θy indicates the rotation angle around the Y-axis, and θz indicates the rotation angle around the Z-axis.

[0063] Furthermore, the first three-dimensional model 521 and the second three-dimensional model 522 are three-dimensional models representing the same object. However, the times (first time T1 and second time T2) at which the shapes of the object were measured are different between the first three-dimensional model 521 and the second three-dimensional model 522. Therefore, if part or all of the object changes over time, the form of the first three-dimensional model 521 and the form of the second three-dimensional model 522 representing the object may differ. The form of the object is, for example, the shape, pattern, or color of part or all of the object, or a combination thereof.

[0064] 3 and 4, compared to the object (first three-dimensional model 521) at the first time T1, the upper half of the object (second three-dimensional model 522) at the second time T2 has disappeared. However, for portions of the object that have not changed between the first time T1 and the second time T2 (e.g., the bottom), the position of the corresponding portion of the first three-dimensional model 521 (e.g., the bottom 521a) and the position of the corresponding portion of the second three-dimensional model 522 (e.g., the bottom 522a) are the same in the three-dimensional coordinate system CS.

[0065] As shown in FIG. 4(b), the model generation unit 11 sets a second rectangular parallelepiped 72 for the second three-dimensional model 522 in the second virtual space VS2. The second rectangular parallelepiped 72 is a defined rectangular parallelepiped similar to the first rectangular parallelepiped 71. As shown in FIGS. 4(a) and 4(b), the centers of the defined rectangular parallelepipeds (the center 74 of the first rectangular parallelepiped 71 and the center 76 of the second rectangular parallelepiped 72) are located inside the three-dimensional models (the first three-dimensional model 521 and the second three-dimensional model 522).

[0066] Next, the difference calculation unit 14 will be described with reference to Fig. 2 and Fig. 5. Fig. 5(a) is a diagram showing an example of a first two-dimensional image 110. Fig. 5(b) is a diagram showing an example of a second two-dimensional image 120.

[0067] 5(a), the first two-dimensional image 110 includes an image 111 and an image 112. As shown in FIG. 5(b), the second two-dimensional image 120 includes an image 121 and an image 122.

[0068] The image 111 and the image 121 are images of the same object. The image 112 is present in the first two-dimensional image 110 but not in the second two-dimensional image 120. The image 122 is not present in the first two-dimensional image 110 but is present in the second two-dimensional image 120.

[0069] The difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120. Then, the difference calculation unit 14 outputs difference data 130 indicating the difference. The storage unit 50 stores the difference data 130. FIG. 5(c) is a diagram illustrating the concept of the difference data 130. As shown in FIG. 5(c), the difference data 130 includes difference data 1120 (dense dots) corresponding to the image 112 in FIG. 5(a), difference data 1220 (sparse dots) corresponding to the image 122 in FIG. 5(b), and difference data 1000. The difference data 1000 is data indicating that there is no difference.

[0070] Specifically, the difference calculation unit 14 calculates the difference for each pixel in the first two-dimensional image 110 and the second two-dimensional image 120. That is, the difference calculation unit 14 calculates the difference in pixel values ​​between corresponding pixels in the first two-dimensional image 110 and the second two-dimensional image 120. More specifically, one pixel is made up of N pixel elements, where N is an integer greater than or equal to 2. Therefore, the pixel value of one pixel includes N pixel element values. Therefore, the difference calculation unit 14 calculates the difference in pixel element values ​​for each pixel element value between corresponding pixels. Therefore, the difference data 130 is a collection of differences (difference values) between the pixel element values ​​of each pixel.

[0071] For example, N=3. In this case, the N pixel elements constituting one pixel are, for example, an R (red) element, a G (green) element, and a B (blue) element. In this case, the three pixel element values ​​included in the pixel value of one pixel are an R value, a G value, and a B value. The R value, the G value, and the B value indicate their respective luminance values. Therefore, the difference calculation unit 14 calculates the difference in the R value, the difference in the G value, and the difference in the B value between corresponding pixels in the first two-dimensional image 110 and the second two-dimensional image 120.

[0072] For example, the difference calculation unit 14 may calculate the difference by subtracting the second two-dimensional image 120 from the first two-dimensional image 110, or may calculate the difference by subtracting the first two-dimensional image 110 from the second two-dimensional image 120.

[0073] Specifically, for example, the difference calculation unit 14 may calculate the difference by subtracting the corresponding pixel element value of the second two-dimensional image 120 from the pixel element value of the first two-dimensional image 110, or may calculate the difference by subtracting the corresponding pixel element value of the first two-dimensional image 110 from the pixel element value of the second two-dimensional image 120.

[0074] Furthermore, the difference calculation section 14 may execute the following process on the difference data 130 to generate new difference data (hereinafter, "difference data 130A").

[0075] That is, for each pixel, if the absolute value of the difference between at least one pixel element value is equal to or greater than the threshold value TH and the difference is a positive value, the difference calculation unit 14 sets a first predetermined value as the new difference corresponding to the pixel. Furthermore, for each pixel, if the absolute value of the difference between at least one pixel element value is equal to or greater than the threshold value TH and the difference is a negative value, the difference calculation unit 14 sets a second predetermined value as the new difference corresponding to the pixel. For each pixel, if the absolute value of the difference between N pixel element values ​​is less than the threshold value TH, the difference calculation unit 14 sets a third predetermined value as the new difference corresponding to the pixel.

[0076] The first predetermined value, the second predetermined value, and the third predetermined value are different from one another. As a result, the difference data 130A is a collection of differences (difference values) newly obtained for each pixel. The storage unit 50 stores the difference data 130A. For example, as shown in FIG. 5(c), the difference data 130A is composed of difference data 1120A having the first predetermined value, difference data 1220A having the second predetermined value, and difference data 1000A having the third predetermined value.

[0077] 5, according to this embodiment, the difference between the two-dimensional images (the first two-dimensional image 110 and the second two-dimensional image 120) is calculated rather than directly analyzing the three-dimensional models (the first three-dimensional model 521 and the second three-dimensional model 522) to calculate the difference. Therefore, the processing load on the server 1 when calculating the difference can be reduced.

[0078] For reference, when the entire area of ​​three-dimensional data (point cloud data) constituting a three-dimensional model is analyzed using an octree algorithm to calculate the difference, the processing load on the computer is large. In contrast, in this embodiment, the difference between two-dimensional images (first two-dimensional image 110 and second two-dimensional image 120) is calculated, thereby reducing the processing load on the server 1 and indirectly calculating the difference between three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522).

[0079] 3, the pre-processing for calculating the difference includes a process of generating a first two-dimensional image 110 that represents a first three-dimensional model 521 on a two-dimensional plane from a first viewpoint 61, and a process of generating a second two-dimensional image 120 that represents a second three-dimensional model 522 on a two-dimensional plane from a second viewpoint 81 that is the same as the first viewpoint 61. In this way, by performing simple pre-processing, the first two-dimensional image 110 and the second two-dimensional image 120 that are the target of difference calculation can be easily calculated.

[0080] 2 may perform an adjustment process on at least one of the first two-dimensional image 110 and the second two-dimensional image 120. In this case, the difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120 after performing the adjustment process.

[0081] The adjustment process includes at least one of a first adjustment process and a second adjustment process.

[0082] The first adjustment process is a process of adjusting the color tone of at least one of the first two-dimensional image 110 and the second two-dimensional image 120. The color tone is the brightness and / or contrast of the image. The image adjustment unit 13, for example, adjusts the color tone of one of the first two-dimensional image 110 and the second two-dimensional image 120 to be closer to or match the color tone of the other image.

[0083] For example, if the first time T1 is noon and the second time T2 is evening, there is a possibility that the difference will be large for all pixels. In this case, even if the object has not changed between the first time T1 and the second time T2, there is a possibility that the object will be determined to have changed because the difference is large. Therefore, by making the color tones of the first 2D image 110 and the second 2D image 120 closer to each other or matching them and then calculating the difference, it is possible to accurately detect the change in the object.

[0084] The second adjustment process is a process of excluding, from the difference target, at least one pixel element of the N pixel elements constituting each pixel of the first two-dimensional image 110 and the second two-dimensional image 120. For example, the image adjustment unit 13 excludes, from the difference target to be calculated by the difference calculation unit 14, a pixel element that depends on the surrounding environment of the object, from the N pixel elements constituting each pixel.

[0085] For example, among the R, G, and B values, the B value may be larger in the evening compared to noon. In this case, even if the object has not changed between the first time T1 and the second time T2, the difference in the B value is large, so it may be determined that the object has changed. Therefore, for example, if the first time T1 is noon and the second time T2 is evening, the B element of the R, G, and B elements of each pixel is excluded from the difference targets before calculating the difference, thereby enabling accurate detection of a change in the object.

[0086] As described above, by providing the image adjustment unit 13, even if the surrounding environment of the object at the first time T1 when the object corresponding to the first three-dimensional model 521 is measured is different from the surrounding environment of the object at the second time T2 when the object corresponding to the second three-dimensional model 522 is measured, the difference between the first two-dimensional image 110 and the second two-dimensional image 120 can be calculated to accurately detect the difference between the object at the first time T1 and the object at the second time T2.

[0087] The first adjustment process and the second adjustment process are merely examples, and any adjustment process may be performed on the first two-dimensional image 110 and the second two-dimensional image 120.

[0088] In this embodiment, the image enhancing unit 15 of FIG. 2 may enhance, in the two-dimensional image or three-dimensional model, areas where the first two-dimensional image 110 and the second two-dimensional image 120 differ based on the difference between the first two-dimensional image 110 and the second two-dimensional image 120. In this case, when a user visually recognizes the two-dimensional image or three-dimensional model, the areas where the first two-dimensional image 110 and the second two-dimensional image 120 differ can be easily identified. For example, the image enhancing unit 15 adds a specific color to areas where the first two-dimensional image 110 and the second two-dimensional image 120 differ in the two-dimensional image or three-dimensional model based on the difference between the first two-dimensional image 110 and the second two-dimensional image 120.

[0089] 6(a) is a diagram showing an example of a two-dimensional image 140 including regions 1221 and 1121 where the first two-dimensional image 110 and the second two-dimensional image 120 differ. As shown in FIG. 6(a), the image enhancing unit 15 enhances regions 1121 and 1221 in the two-dimensional image 140 based on the difference data 130 or difference data 130A (FIG. 5(c)), relative to the region 141 where the first two-dimensional image 110 and the second two-dimensional image 120 do not differ. The region 1121 is identified based on the difference data 1120 or difference data 1120A (FIG. 5(c)). The region 1221 is identified based on the difference data 1220 or difference data 1220A.

[0090] For example, the area 1121 is highlighted by adding a first specific color 1122 (hatched with diagonal lines extending to the upper left). For example, the area 1221 is highlighted by adding a second specific color 1222 (hatched with diagonal lines extending to the upper right).

[0091] The first specific color 1122 is different from the second specific color 1222. Therefore, the user can easily distinguish between an area 1121 that exists in the first two-dimensional image 110 but not in the second two-dimensional image 120 and an area 1221 that does not exist in the first two-dimensional image 110 but exists in the second two-dimensional image 120.

[0092] As an example, the image enhancing unit 15 generates the two-dimensional image 140 shown in FIG. 6(a) based on the difference data 130 or difference data 130A shown in FIG. 5(c) using the first two-dimensional image 110 or the second two-dimensional image 120 as a base image. In this case, for example, the image enhancing unit 15 adds a first specific color 1122 to a region specified by the difference data 1120, 1120A in the first two-dimensional image 110 (FIG. 5(a)) or the second two-dimensional image 120 (FIG. 5(b)), and adds a second specific color 1222 to a region specified by the difference data 1220, 1220A. As a result, in the two-dimensional image 140, the regions 1121, 1221 where the first two-dimensional image 110 and the second two-dimensional image 120 differ are enhanced.

[0093] 2 stores the two-dimensional image 140. Then, the display control unit 16 causes the terminal 2 to display the two-dimensional image 140 via the communication unit 40. As a result, the user of the terminal 2 can easily recognize areas 1121 and 1221 where the first two-dimensional image 110 and the second two-dimensional image 120 differ.

[0094] FIG. 6(b) is a diagram illustrating an example of a three-dimensional model 140A including regions 1221A and 1121A where the first two-dimensional image 110 and the second two-dimensional image 120 differ. As shown in FIG. 6(b), the three-dimensional model 140A is placed in a virtual space VS in which a three-dimensional coordinate system CS is set. The image enhancing unit 15 enhances regions 1121A and 1221A in the three-dimensional model 140A based on the difference data 130 or the difference data 130A (FIG. 5(c)), relative to the region 141A where the first two-dimensional image 110 and the second two-dimensional image 120 do not differ. The region 1121A is identified based on the difference data 1120 or the difference data 1120A (FIG. 5(c)), and is enhanced by adding a first specific color 1122. The area 1221A is specified based on the difference data 1220 or the difference data 1220A, and is highlighted by adding a second specific color 1222.

[0095] As an example, the image enhancing unit 15 uses the first three-dimensional model 521 corresponding to the first two-dimensional image 110 or the second three-dimensional model 522 corresponding to the second two-dimensional image 120 as a base model, and generates the three-dimensional model 140A shown in Fig. 6(b) based on the difference data 130 or the difference data 130A shown in Fig. 5(c). In this case, surfaces are assigned in advance to the first three-dimensional model 521 and the second three-dimensional model 522, and material information is added to the surfaces.

[0096] Therefore, for example, the image enhancing unit 15 adds a first specific color 1122 instead of material information to the region specified by the difference data 1120, 1120A in the first three-dimensional model 521 or the second three-dimensional model 522. Furthermore, the image enhancing unit 15 adds a second specific color 1222 instead of material information to the region specified by the difference data 1220, 1220A in the first three-dimensional model 521 or the second three-dimensional model 522. As a result, the regions 1121A, 1221A in the three-dimensional model 140A where the first two-dimensional image 110 and the second two-dimensional image 120 differ are enhanced.

[0097] 2 stores the three-dimensional model 140A. Then, the display control unit 16 causes the terminal 2 to display the three-dimensional model 140A via the communication unit 40. As a result, the user of the terminal 2 can easily recognize areas 1121A and 1221A on the three-dimensional model 140A where the first two-dimensional image 110 and the second two-dimensional image 120 differ.

[0098] The image generation unit 12 will now be described with reference to FIGS. 2 and 3. The image generation unit 12 may set a plurality of different first viewpoints 61 in a first virtual space VS1 (FIG. 3(a)). Then, the image generation unit 12 may generate a first two-dimensional image 110 for each of the different first viewpoints 61. Furthermore, the image generation unit 12 may set a plurality of different second viewpoints 81 in a second virtual space VS2 (FIG. 3(b)). Then, the image generation unit 12 may generate a second two-dimensional image 120 for each of the different second viewpoints 81. Furthermore, the difference calculation unit 14 may calculate a plurality of differences corresponding to the plurality of first viewpoints 61 and the plurality of second viewpoints 81. Hereinafter, a first viewpoint 61 and a second viewpoint 81 that is the same as the first viewpoint 61 will be referred to as a "viewpoint pair." In this case, a first two-dimensional image 110 and a second two-dimensional image 120 are generated for each different viewpoint pair. Therefore, a difference between the first 2D image 110 and the second 2D image 120 is generated for each different pair of viewpoints. That is, a plurality of differences are generated for each of a plurality of pairs of viewpoints. Therefore, the difference between the object at the first time T1 and the object at the second time T2 can be detected with high accuracy across the entire area of ​​the object.

[0099] For example, in a first virtual space VS1 shown in Fig. 3(a), the image generation unit 12 sets a plurality of first viewpoints 61 at a plurality of points selected from among vertices a1 to a8, midpoints b1 to b12, and centers c1 to c6. Then, in a second virtual space VS2 shown in Fig. 3(b), the image generation unit 12 sets a plurality of second viewpoints 81 that are respectively the same as the plurality of first viewpoints 61.

[0100] Next, an image analysis method according to this embodiment will be described with reference to FIGS. 2 and 7. FIG. 7 is a flowchart showing the image analysis method. The image analysis method is executed by the server 1. As shown in FIG. 7, the image analysis method includes steps S1 to S12. A computer program stored in the storage unit 50 causes the calculation unit 10 to execute steps S1 to S12. In other words, the computer program product realizes steps S1 to S12 when the computer program is executed by the calculation unit 10. The calculation unit 10 corresponds to an example of a "computer" in the present disclosure.

[0101] 2 and 7, first, in step S1, the model generation unit 11 acquires a first three-dimensional model 521. Specifically, the model generation unit 11 generates a first three-dimensional model 521 representing an object at a first time T1 based on a plurality of two-dimensional images generated by capturing images of the object from a plurality of different imaging positions at a first time T1. Alternatively, the model generation unit 11 sets three-dimensional data 513 based on measurement results of the object at the first time T1 in the first three-dimensional model 521. Then, the model generation unit 11 sets a first rectangular parallelepiped 71 surrounding the first three-dimensional model 521 for the first three-dimensional model 521.

[0102] Next, in step S2, the image generation unit 12 generates a first two-dimensional image 110 that represents the first three-dimensional model 521 on a two-dimensional plane defined by a first viewpoint 61 in the first virtual space VS1. The first viewpoint 61 is set on the first rectangular parallelepiped 71.

[0103] Next, in step S3, the model generation unit 11 acquires a second three-dimensional model 522. Specifically, the model generation unit 11 generates a second three-dimensional model 522 representing the object at the second time T2 based on a plurality of two-dimensional images generated by capturing images of the object from a plurality of different imaging positions at the second time T2. Alternatively, the model generation unit 11 sets three-dimensional data 513 based on the measurement results of the object at the second time T2 in the second three-dimensional model 522. Then, the model generation unit 11 sets a second rectangular parallelepiped 72 surrounding the second three-dimensional model 522 for the second three-dimensional model 522. In the present disclosure, setting the second rectangular parallelepiped 72 is not essential.

[0104] Next, in step S4, the image generation unit 12 generates a second two-dimensional image 120 that represents the second three-dimensional model 522 on a two-dimensional plane based on a second viewpoint 81 in the second virtual space VS2. In the three-dimensional coordinate system CS set in the first virtual space VS1 and the second virtual space VS2, the position and line of sight 82 of the second viewpoint 81 are the same as the position and line of sight 82 of the first viewpoint 61, respectively.

[0105] Next, in step S5, the image adjuster 13 performs an adjustment process on at least one of the first two-dimensional image 110 and the second two-dimensional image 120.

[0106] Next, in step S6, the difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120 after the adjustment process is performed, and generates difference data 130.

[0107] Next, in step S7, the image enhancing unit 15 enhances the different area between the first two-dimensional image 110 and the second two-dimensional image 120 in the two-dimensional image 140 or the three-dimensional model 140A based on the difference indicated by the difference data .

[0108] Next, in step S8, the image generating unit 12 determines whether or not the difference data 130 indicating the difference has been generated for all pairs of viewpoints (first viewpoint 61 and second viewpoint 81).

[0109] If a negative determination is made in step S8 (NO), the process proceeds to step S9.

[0110] Next, in step S9, the image generator 12 changes the viewpoint pair (first viewpoint 61 and second viewpoint 81).

[0111] Next, in step S10, the image generating unit 12 generates a first two-dimensional image 110 that represents the first three-dimensional model 521 on a two-dimensional plane based on the changed first viewpoint 61.

[0112] Next, in step S11, the image generation unit 12 generates a second two-dimensional image 120 that represents the second three-dimensional model 522 on a two-dimensional plane based on the changed second viewpoint 81. Then, the process proceeds to step S5. Steps S5 to S11 are repeated until difference data 130 is calculated for all pairs of viewpoints (first viewpoint 61 and second viewpoint 81).

[0113] On the other hand, if the determination in step S8 is affirmative (YES), the process proceeds to step S12.

[0114] Next, in step S12, in response to a request from the user's terminal 2, the display control unit 16 causes the terminal 2 to display, via the communication unit 40, the two-dimensional image 140 or the three-dimensional model 140A in which the different areas between the first two-dimensional image 110 and the second two-dimensional image 120 are emphasized. Then, the image analysis method ends.

[0115] 7, according to the image analysis method of this embodiment, the difference between two-dimensional images (first two-dimensional image 110 and second two-dimensional image 120) is calculated rather than directly analyzing three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522) to calculate the difference. Therefore, the processing load on server 1 when calculating the difference can be reduced.

[0116] Although the preferred embodiments and modifications of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0117] The devices or systems described herein may be realized as a single device, or may be realized by multiple devices (e.g., cloud servers) partially or entirely connected via a network. For example, some or all of the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may be realized by the same computer or server. For example, the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may each be realized by a separate computer or server. Furthermore, for example, the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may be realized by the terminal 2 or the three-dimensional measurement device 7. For example, the video data 511, the still image data set 512, the three-dimensional data 513, the first three-dimensional model 521, the second three-dimensional model 522, the first two-dimensional image 110, the second two-dimensional image 120, the difference data 130, 130A, the two-dimensional image 140, and the three-dimensional model 140A may each be stored in a separate storage device or server.

[0118] The series of processes performed by the device described herein may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the calculation unit 10 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium storing such a computer program may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via a network, without using a recording medium.

[0119] Furthermore, the processes described herein using flowchart diagrams do not necessarily have to be performed in the order shown. Some process steps may be performed in parallel. Additional process steps may be employed, and some process steps may be omitted.

[0120] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0121] The following configurations also fall within the technical scope of the present disclosure.

[0122] (Item 1) an image generation unit that generates a first two-dimensional image and a second two-dimensional image; a difference calculation unit that calculates a difference between the first two-dimensional image and the second two-dimensional image, The image generation unit generating a first two-dimensional image that represents a first three-dimensional model generated based on a measurement result of the shape of the object in real space at a first time on a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image that represents a second three-dimensional model generated based on a measurement result of the shape of the object in the real space at a second time on a two-dimensional plane from a second viewpoint in a second virtual space; The same coordinate system is set in the first virtual space and the second virtual space, In the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

[0123] (Item 2) Item 2. The image analyzing device according to item 1, wherein the first viewpoint is set on a rectangular parallelepiped that surrounds the first three-dimensional model.

[0124] (Item 3) 3. The image analyzing device according to item 2, wherein the first viewpoint is set at a vertex, a midpoint of a side, or a center of a face of the rectangular parallelepiped.

[0125] (Item 4) 3. The image analysis device according to item 2, wherein the first viewpoint is set at a vertex of the rectangular parallelepiped and faces in the direction of a diagonal line extending from the vertex.

[0126] (Item 5) The image generation unit a plurality of different first viewpoints are set in the first virtual space; generating the first two-dimensional image for each of the different first viewpoints; a plurality of different second viewpoints are set in the second virtual space; generating the second two-dimensional image for each of the different second viewpoints; 5. The image analyzing device according to claim 1, wherein the difference calculation unit calculates a plurality of the differences corresponding to the plurality of first viewpoints and the plurality of second viewpoints.

[0127] (Item 6) further comprising an image adjustment unit that performs an adjustment process on at least one of the first two-dimensional image and the second two-dimensional image; the adjustment process includes at least one of a first adjustment process and a second adjustment process; the first adjustment process is a process of adjusting a color tone of the at least one image, the second adjustment process is a process of excluding at least one pixel element from a difference target among a plurality of pixel elements constituting each pixel of the first two-dimensional image and the second two-dimensional image, 6. The image analyzing device according to any one of items 1 to 5, wherein the difference calculation unit calculates the difference between the first two-dimensional image and the second two-dimensional image after the adjustment process is performed.

[0128] (Item 7) 7. The image analyzing device according to claim 1, further comprising an image enhancing unit that enhances, in a two-dimensional image or a three-dimensional model, a region where the first two-dimensional image and the second two-dimensional image differ based on the difference.

[0129] (Item 8) generating a first two-dimensional image representing a first three-dimensional model generated based on a measurement result of the shape of the object in real space at a first time on a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in the real space at a second time on a two-dimensional plane from a second viewpoint in a second virtual space; calculating a difference between the first two-dimensional image and the second two-dimensional image; The same coordinate system is set in the first virtual space and the second virtual space, An image analysis method, wherein in the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

[0130] (Item 9) On the computer, generating a first two-dimensional image representing a first three-dimensional model generated based on a measurement result of the shape of the object in real space at a first time on a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in the real space at a second time on a two-dimensional plane from a second viewpoint in a second virtual space; calculating a difference between the first two-dimensional image and the second two-dimensional image; the same three-dimensional coordinate system is set in the first virtual space and the second virtual space, In the three-dimensional coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively. [Industrial Applicability]

[0131] The present disclosure provides an image analysis device, an image analysis method, and a computer program, and has industrial applicability. [Explanation of symbols]

[0132] 1 server (image analysis device), 11 model generation unit, 12 image generation unit, 13 image adjustment unit, 14 difference calculation unit, 15 image enhancement unit, 16 display control unit

Claims

1. an image generation unit that generates a first two-dimensional image and a second two-dimensional image; a difference calculation unit that calculates a difference between the first two-dimensional image and the second two-dimensional image, The image generation unit generating a first two-dimensional image that represents a first three-dimensional model generated based on a measurement result of the shape of the object at a first time in real space on a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on a measurement result of the shape of the object in the real space at a second time on a two-dimensional plane from a second viewpoint in a second virtual space; the same coordinate system is set in the first virtual space and the second virtual space, In the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively; the first viewpoint is set on a rectangular parallelepiped surrounding the first three-dimensional model; The rectangular parallelepiped defines a display range when the first three-dimensional model is displayed on a viewer.

2. The image analysis device according to claim 1 , wherein the first viewpoint is set at a vertex, a midpoint of a side, or a center of a face of the rectangular parallelepiped.

3. The image analysis device according to claim 1 , wherein the first viewpoint is set at a vertex of the rectangular parallelepiped and faces in a direction of a diagonal line extending from the vertex.

4. The image generation unit a plurality of different first viewpoints are set in the first virtual space; generating the first two-dimensional image for each of the different first viewpoints; a plurality of different second viewpoints are set in the second virtual space; generating the second two-dimensional image for each of the different second viewpoints; The image analysis device according to claim 1 , wherein the difference calculation unit calculates a plurality of the differences corresponding to the plurality of first viewpoints and the plurality of second viewpoints.

5. an image adjustment unit that performs an adjustment process on at least one of the first two-dimensional image and the second two-dimensional image; the adjustment process includes at least one of a first adjustment process and a second adjustment process, the first adjustment process is a process of adjusting a color tone of the at least one image, the second adjustment process is a process of excluding at least one pixel element from a difference target among a plurality of pixel elements constituting each pixel of the first two-dimensional image and the second two-dimensional image, The image analyzing device according to claim 1 , wherein the difference calculation unit calculates the difference between the first two-dimensional image and the second two-dimensional image after the adjustment process is performed.

6. 3. The image analysis device according to claim 1, further comprising an image enhancement unit that enhances, in a two-dimensional image or a three-dimensional model, an area where the first two-dimensional image and the second two-dimensional image differ based on the difference.

7. generating a first two-dimensional image representing a first three-dimensional model generated based on a measurement result of the shape of the object in real space at a first time on a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in the real space at a second time on a two-dimensional plane from a second viewpoint in a second virtual space; calculating a difference between the first two-dimensional image and the second two-dimensional image, the same coordinate system is set in the first virtual space and the second virtual space, In the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively; the first viewpoint is set on a rectangular parallelepiped surrounding the first three-dimensional model; The rectangular parallelepiped defines a display range when the first three-dimensional model is displayed on a viewer.

8. On the computer, generating a first two-dimensional image representing a first three-dimensional model generated based on a measurement result of the shape of the object in real space at a first time on a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in the real space at a second time on a two-dimensional plane from a second viewpoint in a second virtual space; calculating a difference between the first two-dimensional image and the second two-dimensional image; the same three-dimensional coordinate system is set in the first virtual space and the second virtual space, In the three-dimensional coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively; the first viewpoint is set on a rectangular parallelepiped surrounding the first three-dimensional model; The rectangular parallelepiped defines a display range when the first three-dimensional model is displayed in a viewer.

Images