Image analysis device, image analysis method, and computer program

Abstract

This invention provides an image analysis device, method, and program that reduce the processing load when calculating differences. [Solution] In an image analysis system comprising a terminal, a mobile device, an imaging device, and a three-dimensional measuring device, the server, which is the image analysis device, comprises an image generation unit that generates a first two-dimensional image and a second two-dimensional image, and a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image. The image generation unit generates a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space, and generates a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of an object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space. The first and second virtual spaces are set to the same coordinate system, and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

Description

Technical Field

[0001] The present disclosure relates to an image analysis device, an image analysis method, and a computer program.

Background Art

[0002] The difference detection device described in Patent Document 1 acquires a plurality of images by photographing a structure from a plurality of directions in the first inspection, and generates three-dimensional first point cloud data from the plurality of images. Then, in the second inspection, the difference detection device again acquires a plurality of images obtained by imaging the structure from a plurality of directions, and generates three-dimensional second point cloud data from the plurality of images. The first point cloud data and the second point cloud data indicate the same location of the structure.

[0003] The difference detection device compares the color information indicating the colors of the points included in the first point cloud data with the color information indicating the colors of the points included in the second point cloud data. Then, for a location where the difference between the two is equal to or greater than the threshold, in the second point cloud data, the color of the location where the difference is equal to or greater than the threshold is changed and displayed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the difference detection device described in Patent Document 1, the entire area of the three-dimensional data (the first point cloud data and the second point cloud data) is analyzed to calculate the difference. Therefore, the processing load of the difference detection device was excessive.

[0006] Therefore, this disclosure has been made in view of the above-mentioned problems, and its purpose is to provide an image analysis device, an image analysis method, and a computer program that can reduce the processing load when calculating differences. [Means for solving the problem]

[0007] According to this disclosure, an image analysis device is provided, comprising: an image generation unit that generates a first two-dimensional image and a second two-dimensional image; and a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image, wherein the image generation unit generates a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; and generates a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space, wherein the same coordinate system is set in the first virtual space and the second virtual space, and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0008] Furthermore, the present disclosure provides an image analysis method comprising the steps of: generating a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; generating a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space; and calculating the difference between the first two-dimensional image and the second two-dimensional image, wherein the same coordinate system is set for the first and second virtual spaces, and in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0009] Furthermore, according to this disclosure, a computer program is provided which causes a computer to perform the following steps: generate a first two-dimensional image representing a first three-dimensional model generated based on the measurement results of the shape of an object in real space at a first time in a two-dimensional plane from a first viewpoint in a first virtual space; generate a second two-dimensional image representing a second three-dimensional model generated based on the measurement results of the shape of the object in real space at a second time in a two-dimensional plane from a second viewpoint in a second virtual space; and calculate the difference between the first two-dimensional image and the second two-dimensional image, wherein the same three-dimensional coordinate system is set in the first virtual space and the second virtual space, and in the three-dimensional coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively. [Effects of the Invention]

[0010] According to this disclosure, an image analysis device, an image analysis method, and a computer program can be provided that can reduce the processing load when calculating differences. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing an example configuration of an image analysis system according to one embodiment of the present invention. [Figure 2] This block diagram shows an example of a server configuration according to the same embodiment. [Figure 3] (a) is a schematic perspective view showing the first viewpoint, the first three-dimensional model, and the first rectangular prism in the first virtual space according to the same embodiment. (b) is a schematic perspective view showing the second viewpoint, the second three-dimensional model, and the second rectangular prism in the second virtual space according to the same embodiment. [Figure 4] (a) is a schematic side view showing the first viewpoint, the first three-dimensional model, and the first rectangular prism in the first virtual space according to the same embodiment. (b) is a schematic side view showing the second viewpoint, the second three-dimensional model, and the second rectangular prism in the second virtual space according to the same embodiment. [Figure 5](a) is a diagram showing an example of a first two-dimensional image according to the same embodiment. (b) is a diagram showing an example of a second two-dimensional image according to the same embodiment. (c) is a diagram showing the concept of difference data according to the same embodiment. [Figure 6] (a) is a diagram showing an example of a two-dimensional image that includes a region in which the first two-dimensional image and the second two-dimensional image according to the same embodiment differ. (b) is a diagram showing an example of a three-dimensional model that includes a region in which the first two-dimensional image and the second two-dimensional image according to the same embodiment differ. [Figure 7] This is a flowchart showing the image analysis method according to the present invention. [Modes for carrying out the invention]

[0012] Preferred embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted. In this embodiment, a rectangular parallelepiped includes a cube.

[0013] Figure 1 is a diagram showing an example configuration of an image analysis system SYS according to one embodiment of the present invention. As shown in Figure 1, the image analysis system SYS includes a server 1. Server 1 corresponds to an example of the "image analysis device" in this disclosure.

[0014] Server 1 calculates the difference between two two-dimensional images, each representing the same three-dimensional model on a two-dimensional plane from the same viewpoint, in order to detect the differences between two three-dimensional models generated by measuring the same object at different times. In this way, according to this embodiment, since the difference between two-dimensional images is calculated, the processing load on Server 1 when calculating the difference can be reduced compared to when the entire area of ​​the three-dimensional data constituting the three-dimensional model is analyzed to calculate the difference. Details of this point will be described later.

[0015] Furthermore, the image analysis system SYS comprises at least one terminal 2, at least one mobile device 3, at least one imaging device 5, or at least one three-dimensional measuring device 7.

[0016] Server 1, terminal 2, mobile device 3, imaging device 5, and three-dimensional measurement device 7 are connected to network NW. Network NW includes, for example, the Internet, a closed network, a public telephone network, a LAN (Local Area Network), and a short-range wireless network.

[0017] Terminal 2 is, for example, a personal computer (e.g., a notebook computer, a desktop computer, or a tablet).

[0018] Mobile device 3 is, for example, an unmanned mobile device or a manned mobile device. The unmanned mobile device is, for example, an unmanned aircraft such as a drone, an unmanned ground vehicle, an unmanned underwater vehicle, or an unmanned surface vessel. The unmanned ground vehicle is, for example, an unmanned ground vehicle or an unmanned ground vehicle modeled after a living being (e.g., a snake-shaped unmanned ground vehicle). The manned mobile device is, for example, an aircraft, an automobile, a ship, or a submarine. Mobile device 3 includes camera 4.

[0019] Imaging device 5 includes camera 6. Imaging device 5 is, for example, a mobile terminal such as a smartphone. Imaging device 5 may be, for example, camera 6 itself.

[0020] Cameras 4 and 6 image an object and generate video data showing a video including an image of the object. A video is a set of consecutive two-dimensional images. Cameras 4 and 6 may generate a plurality of still image data respectively showing a plurality of still images including an image of the object. A still image is a two-dimensional image.

[0021] Hereinafter, the video data is described as "video data 511" (FIG. 2). Also, the plurality of still image data is described as "still image data set 512" (FIG. 2).

[0022] Hereinafter, when it is not necessary to distinguish between cameras 4 and 6, cameras 4 and 6 are collectively referred to as "camera CM".

[0023] The three-dimensional measuring device 7 measures the shape of an object and generates three-dimensional data (hereinafter referred to as "three-dimensional data 513") representing the shape of the object. The three-dimensional data 513 is typically point cloud data. A three-dimensional model of the object is formed from the three-dimensional data 513. The three-dimensional measuring device 7 may be contact-type or non-contact-type. Furthermore, the three-dimensional measuring device 7 may be active-type or passive-type. Active methods include, for example, optical radar (ToF (Time of Flight) method), active stereo method, or optical interferometry. Optical radar measures the shape of an object by irradiating it with light and measuring the time or phase change until the reflected light returns to the detector. An example of optical radar is LiDAR (Light Detection And Ranging). Active stereo method measures the shape of an object by projecting a laser beam, slit light, or a code pattern of light. Optical interferometry measures the shape of an object by irradiating it with light and utilizing the interference of light. A passive method is, for example, the lens focusing method.

[0024] Furthermore, the objects to be imaged by the camera CM and the objects to be measured by the three-dimensional measuring device 7 are not particularly limited, as long as they can be imaged by the camera CM or measured by the three-dimensional measuring device 7. Also, for example, the size, shape, pattern, and color of the objects are not particularly limited. For example, an object is one or more movable or immovable property. An object is, for example, one or more objects. Typically, an object is one or more stationary objects. Stationary objects are, for example, man-made or natural objects. Man-made objects are, for example, structures, machinery, electronic equipment, or copyrighted works. Structures are, for example, buildings or infrastructure facilities. Buildings are, for example, office buildings or houses. Infrastructure facilities are facilities for developing social infrastructure. For example, infrastructure facilities are roads, bridges, road traffic facilities, power generation facilities, power transmission facilities, water treatment facilities, or gas distribution facilities. Machinery is, for example, automobiles, work vehicles, trains, aircraft, ships, submarines, or robots. Natural objects include, for example, trees, forests, the ground surface, cliffs, coastlines, or rivers.

[0025] Terminal 2 acquires video data 511 or still image data set 512 generated by camera CM, or three-dimensional data 513 generated by three-dimensional measuring device 7. For example, terminal 2 receives video data 511 or still image data set 512 transmitted from camera CM, or three-dimensional data 513 transmitted from three-dimensional measuring device 7, via network NW. Terminal 2 transmits video data 511, still image data set 512, and three-dimensional data 513 to server 1 via network NW. The mobile device 3 and imaging device 5 may also transmit video data 511 or still image data set 512 to server 1 via network NW. The three-dimensional measuring device 7 may also transmit three-dimensional data 513 to server 1 via network NW.

[0026] Figure 2 is a block diagram showing an example configuration of Server 1 in Figure 1. As shown in Figure 2, Server 1 includes an arithmetic unit 10, a communication unit 40, and a storage unit 50. Server 1 may also include an input unit 20 and a display unit 30.

[0027] The input unit 20 is an input device for inputting various types of information to the calculation unit 10. For example, the input unit 20 may be a keyboard and pointing device, or a touch panel.

[0028] The display unit 30 displays various information. The display unit 30 is, for example, a liquid crystal display or an organic electroluminescent display.

[0029] The communication unit 40 is connected to a network NW. The communication unit 40 communicates with external devices connected to the network NW. The external devices are, for example, a terminal 2, a mobile device 3, an imaging device 5, and a three-dimensional measuring device 7. The communication unit 40 is a communication device that performs communication according to a predetermined communication protocol, and includes, for example, a network interface controller. The predetermined communication protocol is, for example, a protocol compliant with Ethernet® and an Internet Protocol Suite.

[0030] The communication unit 40 receives video data 511 or still image data set 512 from the terminal 2, mobile device 3, and imaging device 5 via the network NW. The communication unit 40 also receives three-dimensional data 513 from the terminal 2 and three-dimensional measuring device 7 via the network NW.

[0031] The storage unit 50 includes a storage device and stores data and computer programs. The storage unit 50 includes a main storage device such as a semiconductor memory and an auxiliary storage device such as a semiconductor memory and a hard disk drive. The storage unit 50 may also include a removable medium such as an optical disc. The storage unit 50 may be, for example, a non-temporary computer-readable storage medium.

[0032] The storage unit 50 stores video data 511, still image data set 512, and three-dimensional data 513. The video data 511, still image data set 512, and three-dimensional data 513 are associated with attribute information (hereinafter referred to as "attribute information AT"). Attribute information AT includes, for example, user information, data acquisition conditions, and object information. User information includes, for example, identification information of the user of terminal 2, mobile device 3, imaging device 5, or three-dimensional measuring device 7. Data acquisition conditions include, for example, imaging time or measurement time. Imaging time and measurement time are indicated by one or more of the year, month, day, and time. Data acquisition conditions may also include information on the imaging location or measurement location. The imaging location and measurement location are indicated, for example, by the position coordinates of camera CM and three-dimensional measuring device 7 acquired by GPS (Global Positioning System) or GNSS (Global Navigation Satellite System), respectively. Object information includes, for example, identification information of the object. The object information may include the coordinates of the ground control point (GCP).

[0033] Furthermore, the memory unit 50 stores the first three-dimensional model 521, the second three-dimensional model 522, the first two-dimensional image 110, the second two-dimensional image 120, the difference data 130 (130A), the two-dimensional image 140, and the three-dimensional model 140A. This data is stored when it is generated. Details of this data will be described later.

[0034] The arithmetic unit 10 performs various calculations. The arithmetic unit 10 includes processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

[0035] Specifically, the calculation unit 10 includes an image generation unit 12 and a difference calculation unit 14. The calculation unit 10 may also include a model generation unit 11, an image adjustment unit 13, an image enhancement unit 15, and a display control unit 16. For example, the calculation unit 10 functions as the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 by executing a computer program stored in the storage unit 50.

[0036] The model generation unit 11 obtains multiple two-dimensional images from the video data 511 or still image dataset 512 of the storage unit 50, which are generated by capturing images of the object from multiple different imaging positions. The model generation unit 11 then generates a three-dimensional model of the object based on the multiple two-dimensional images. The three-dimensional model is placed in a virtual space. The three-dimensional model shows the three-dimensional shape of the object. The three-dimensional model is composed of point cloud data. Point cloud data is data that represents a point cloud. A point cloud is a collection of multiple points. Point cloud data includes the three-dimensional coordinates of each point. Point cloud data may further include one or more pieces of information from the following: color information (e.g., RGB values) of each point, normal vector information of each point, and reflectance information.

[0037] As an example, the model generation unit 11 generates a three-dimensional model by performing SfM (Structure from Motion) processing. SfM processing is a process that generates a three-dimensional model by using the principle of triangulation based on multiple two-dimensional images generated by capturing an object having multiple feature points from multiple imaging positions. SfM processing preferably includes bundle adjustment. Bundle adjustment is a process that minimizes reprojection errors. In addition to SfM processing, the model generation unit 11 may also perform MVS (Multi View Stereo) processing. MVS processing is a process that calculates the depth and normal for each pixel of each two-dimensional image by multi-view stereo measurement, integrates these, and generates a dense point cloud of the object.

[0038] Furthermore, it is preferable for the model generation unit 11 to assign faces to the three-dimensional model based on the point cloud data. The process of assigning faces is, for example, a process of converting the point cloud data constituting the three-dimensional model into mesh data. In this case, for example, the model generation unit 11 generates multiple polygonal faces (e.g., triangles) by connecting points in the point cloud data, and represents the three-dimensional model with these multiple polygonal faces. In this case, for example, the model generation unit 11 generates TIN (Triangulated Irregular Network) data based on the point cloud data, and represents the three-dimensional model with the TIN data. As a result, the three-dimensional model is represented by a set of triangular faces. Note that the process of assigning faces may also be, for example, a process of converting point cloud data into surface data.

[0039] Furthermore, the model generation unit 11 may add material information to the surfaces assigned to the three-dimensional model based on a two-dimensional image. Material information includes information about the color and pattern of the object. For example, the model generation unit 11 may perform a process of mapping a texture to each surface (e.g., each polygon) that constitutes the mesh data of the three-dimensional model.

[0040] As explained above with reference to Figure 2, the three-dimensional model is generated based on the imaging results of the object by the camera CM. Therefore, imaging of the object by the camera CM can be considered as a measurement of the object's shape. Thus, the time spent imaging the object with the camera CM is the same as the time spent measuring the object with the camera CM.

[0041] The three-dimensional data 513 generated by the three-dimensional measuring device 7 is the same as the point cloud data that constitutes the three-dimensional model generated by the model generation unit 11. Similarly, surfaces may be assigned to the three-dimensional model shown by the three-dimensional data 513, and material information may be added to these surfaces.

[0042] The memory unit 50 stores the three-dimensional model generated by the model generation unit 11 as the first three-dimensional model 521. The memory unit 50 also stores another three-dimensional model generated by the model generation unit 11 as the second three-dimensional model 522. Alternatively, three-dimensional data 513 generated by the three-dimensional measurement device 7 may be treated as the first three-dimensional model 521, and another three-dimensional data 513 may be treated as the second three-dimensional model 522. These details will be described later.

[0043] Furthermore, the methods used by the three-dimensional measuring device 7 and SfM processing are given as examples of how to generate the three-dimensional data 513, the first three-dimensional model 521, and the second three-dimensional model 522. However, these generation methods are not particularly limited as long as three-dimensional data or three-dimensional models can be generated. For example, the three-dimensional data or three-dimensional model may be generated by 3D Gaussian splatting or NeRF (Neural Radiance Field). Also, the three-dimensional data or three-dimensional model is not limited to point cloud data, and the data format is not particularly limited.

[0044] Next, the image generation unit 12 will be described with reference to Figures 2 to 4. Figure 3(a) is a schematic perspective view showing the first viewpoint 61, the first three-dimensional model 521, and the first rectangular prism 71 in the first virtual space VS1. Figure 3(b) is a schematic perspective view showing the second viewpoint 81, the second three-dimensional model 522, and the second rectangular prism 72 in the second virtual space VS2. Figure 4(a) is a schematic side view showing the first viewpoint 61, the first three-dimensional model 521, and the first rectangular prism 71 in the first virtual space VS1. Figure 4(a) corresponds to Figure 3(a). Figure 4(b) is a schematic side view showing the second viewpoint 81, the second three-dimensional model 522, and the second rectangular prism 72 in the second virtual space VS2. Figure 4(b) corresponds to Figure 3(b). For the sake of clarity, Figures 3 and 4 show a virtual reference plane 75.

[0045] As shown in Figures 2 and 3(a), the first three-dimensional model 521 is placed in the first virtual space VS1. The first three-dimensional model 521 is a three-dimensional model generated based on the measurement results of the shape of the object in real space at the first time (hereinafter referred to as "first time T1"). The first time T1 is indicated by attribute information AT. For example, the first time T1 is indicated by one or more of the year, month, day, and time.

[0046] A three-dimensional coordinate system CS is set in the first virtual space VS1. The three-dimensional coordinate system CS is defined by mutually orthogonal X, Y, and Z axes. The three-dimensional coordinate system CS may be a coordinate system with a predetermined position in the first virtual space VS1 as its origin O, or it may be a coordinate system to which geospatial coordinates (ground coordinates) or actual size information is assigned. In Figures 3 and 4, for the sake of explanation, as an example, the origin O of the three-dimensional coordinate system CS is set on a virtual reference plane 75. The three-dimensional coordinate system CS corresponds to an example of a "coordinate system" in this disclosure.

[0047] The image generation unit 12 generates a first two-dimensional image 110, which represents the first three-dimensional model 521 on a two-dimensional plane from a first viewpoint 61 in the first virtual space VS1. The first viewpoint 61 is oriented in the line of sight direction 62 at position p1 in the first virtual space VS1. In other words, the image generation unit 12 generates the first two-dimensional image 110 by capturing the first three-dimensional model 521 from the first viewpoint 61. To put it another way, the image generation unit 12 generates the first two-dimensional image 110 by simulating imaging the first three-dimensional model 521 with a first virtual camera 63 having the first viewpoint 61. The storage unit 50 stores the first two-dimensional image 110.

[0048] In detail, the model generation unit 11 sets a first rectangular prism 71 for the first three-dimensional model 521 in the first virtual space VS1. The first rectangular prism 71 surrounds the first three-dimensional model 521. The first rectangular prism 71 has a shape and size corresponding to the shape and size of the first three-dimensional model 521. The first rectangular prism 71 is a rectangular prism that defines the display range when the first three-dimensional model 521 is displayed on a viewer. Hereinafter, the rectangular prism that defines the display range when a three-dimensional model is displayed on a viewer may be referred to as a "defining rectangular prism". A viewer is software or a device for viewing images. A defining rectangular prism is, for example, a bounding box. The first rectangular prism 71 corresponds to an example of a "rectangular prism" in this disclosure.

[0049] The first viewpoint 61 is set on the first rectangular prism 71. Therefore, according to this embodiment, the first viewpoint 61 can be set more easily compared to the case where the first viewpoint 61 is set at any point in the first virtual space VS1. In particular, since the first rectangular prism 71 is always set when the first three-dimensional model 521 is displayed in the viewer, it is possible to suppress the occurrence of additional processing solely for the purpose of determining the first viewpoint 61.

[0050] Preferably, the first viewpoint 61 is set to any of the eight vertices a1, a2, a3, a4, a5, a6, a7, a8 of the first rectangular prism 71, any of the midpoints b1, b2, b3, b4, b5, b6, b7, b8, b9, b10, b11, b12 of the twelve edges of the first rectangular prism 71, or any of the centers c1, c2, c3, c4, c5, c6 of the six faces of the first rectangular prism 71. Thus, according to this preferred example, the first viewpoint 61 can be set more easily.

[0051] For example, if the first viewpoint 61 is set to one of the vertices a1 to a8, the line of sight 62 of the first viewpoint 61 will point in the direction of the diagonal of the first cuboid 71. In other words, the line of sight 62 will point to the vertex diagonally opposite to the vertex where the first viewpoint 61 is set.

[0052] If the first viewpoint 61 is set to one of the vertices a1 to a8, then, for example, a first two-dimensional image 110 can be generated when viewing the first three-dimensional model 521 from diagonally above or diagonally below. In particular, since the positions of vertices a1 to a8 are uniquely determined, the calculation process for the positions of vertices a1 to a8 can be omitted.

[0053] In the example shown in Figure 3(a), the position p1 of the first viewpoint 61 is vertex a1. Specifically, the first viewpoint 61 is set to vertex a1 of the first rectangular prism 71 and faces in the direction of the diagonal 73 extending from vertex a1. In other words, the line of sight 62 is directed along the diagonal 73, from vertex a1 to vertex a7.

[0054] For example, if the first viewpoint 61 is set to one of the midpoints b1 to b12, the line of sight direction 62 of the first viewpoint 61 will point diagonally opposite the midpoint where the first viewpoint 61 is set, across the first three-dimensional model 521. In this case, for example, if the first viewpoint 61 is set to midpoint b1, the line of sight direction 62 will point from midpoint b1 to midpoint b6.

[0055] If the first viewpoint 61 is set to one of the midpoints b1 to b12, then, for example, a first two-dimensional image 110 can be generated when viewing the first three-dimensional model 521 from diagonally above, diagonally below, or diagonally to the side.

[0056] For example, if the first viewpoint 61 is set to one of the centers c1 to c6, the line of sight direction 62 of the first viewpoint 61 will point toward the center opposite to the center where the first viewpoint 61 is set, in a direction perpendicular to the plane containing the center where the first viewpoint 61 is set. In this case, for example, if the first viewpoint 61 is set to center c1, the line of sight direction 62 will point from center c1 toward center c3.

[0057] If the first viewpoint 61 is set to one of the centers c1 to c6, for example, a first two-dimensional image 110 can be generated as if the first three-dimensional model 521 were viewed from a direction perpendicular to the face of the first rectangular prism 71.

[0058] On the other hand, as shown in Figure 3(b), the second three-dimensional model 522 is placed in the second virtual space VS2. The second three-dimensional model 522 is a three-dimensional model generated based on the measurement results of the shape of the object in real space at the second time (hereinafter, "second time T2"). The second time T2 is indicated by attribute information AT. For example, the second time T2 is indicated by one or more of the year, month, day, and time. The second time T2 is different from the first time T1. For example, the second time T2 is a time earlier than the first time T1.

[0059] The second virtual space VS2 is assigned the same three-dimensional coordinate system CS. In other words, the second virtual space VS2 and the first virtual space VS1 are assigned the same three-dimensional coordinate system CS.

[0060] The image generation unit 12 generates a second two-dimensional image 120, which represents the second three-dimensional model 522 on a two-dimensional plane from a second viewpoint 81 in the second virtual space VS2. The second viewpoint 81 is facing the line of sight direction 82 at position p2 in the second virtual space VS2. In other words, the image generation unit 12 generates the second two-dimensional image 120 by capturing the second three-dimensional model 522 from the second viewpoint 81. To put it another way, the image generation unit 12 generates the second two-dimensional image 120 by simulating imaging the second three-dimensional model 522 with a second virtual camera 83 having the second viewpoint 81. The storage unit 50 stores the second two-dimensional image 120.

[0061] In particular, as shown in Figures 3(a) and 3(b), in the three-dimensional coordinate system CS, the position p2 and line of sight direction 82 of the second viewpoint 81 (second virtual camera 83) are the same as the position p1 and line of sight direction 62 of the first viewpoint 61 (first virtual camera 63), respectively.

[0062] Therefore, as shown in Figures 4(a) and 4(b), the three-dimensional coordinates (aX, aY, aZ) indicating the position p2 of the second viewpoint 81 are the same as the three-dimensional coordinates (aX, aY, aZ) indicating the position p1 of the first viewpoint 61. Also, the line-of-sight directions 62 and 82 (the orientations of the first virtual camera 63 and the second virtual camera 83) are represented by the same rotation angles (θx, θy, θz) with respect to the reference direction (reference orientation). θx represents the rotation angle around the X axis, θy represents the rotation angle around the Y axis, and θz represents the rotation angle around the Z axis.

[0063] Furthermore, the first three-dimensional model 521 and the second three-dimensional model 522 are three-dimensional models representing the same object. However, the time periods (first time T1 and second time T2) at which the shape of the object was measured are different for the first three-dimensional model 521 and the second three-dimensional model 522. Therefore, if part or all of the object changes over time, the form of the first three-dimensional model 521 and the form of the second three-dimensional model 522 representing the object may differ. The form of the object may be, for example, the shape, pattern, or color of part or all of the object, or a combination thereof.

[0064] In the examples in Figures 3 and 4, the upper half of the object (first three-dimensional model 521) at time T1 has disappeared in the object (second three-dimensional model 522) at time T2. However, for the part of the object that has not changed between time T1 and time T2 (e.g., the bottom), the position of the corresponding part of the first three-dimensional model 521 (e.g., bottom 521a) and the position of the corresponding part of the second three-dimensional model 522 (e.g., bottom 522a) are the same in the three-dimensional coordinate system CS.

[0065] As shown in Figure 4(b), the model generation unit 11 sets up a second rectangular prism 72 for the second three-dimensional model 522 in the second virtual space VS2. The second rectangular prism 72 is a defined rectangular prism similar to the first rectangular prism 71. As shown in Figures 4(a) and 4(b), the centers of the defined rectangular prisms (center 74 of the first rectangular prism 71 and center 76 of the second rectangular prism 72) are located inside the three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522).

[0066] Next, the difference calculation unit 14 will be described with reference to Figures 2 and 5. Figure 5(a) shows an example of the first two-dimensional image 110. Figure 5(b) shows an example of the second two-dimensional image 120.

[0067] As shown in Figure 5(a), the first two-dimensional image 110 includes images 111 and 112. As shown in Figure 5(b), the second two-dimensional image 120 includes images 121 and 122.

[0068] Images 111 and 121 are images of the same object. Image 112 exists in the first two-dimensional image 110 but not in the second two-dimensional image 120. Image 122 does not exist in the first two-dimensional image 110 but does exist in the second two-dimensional image 120.

[0069] The difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120. The difference calculation unit 14 then outputs difference data 130 indicating the difference. The storage unit 50 stores the difference data 130. Figure 5(c) is a diagram illustrating the concept of difference data 130. As shown in Figure 5(c), the difference data 130 includes difference data 1120 (dense dots) corresponding to the image 112 in Figure 5(a), difference data 1220 (sparse dots) corresponding to the image 122 in Figure 5(b), and difference data 1000. Difference data 1000 is data indicating that there is no difference.

[0070] Specifically, the difference calculation unit 14 calculates the difference for each pixel of the first two-dimensional image 110 and the second two-dimensional image 120. In other words, the difference calculation unit 14 calculates the difference in pixel values ​​between corresponding pixels in the first two-dimensional image 110 and the second two-dimensional image 120. More specifically, one pixel is composed of N pixel elements, where N is an integer greater than or equal to 2. Therefore, the pixel value of one pixel contains the values ​​of N pixel elements. Thus, the difference calculation unit 14 calculates the difference in pixel element values ​​for each pixel between corresponding pixels. For this reason, the difference data 130 is a set of differences (difference values) in the pixel element values ​​of each pixel.

[0071] For example, N=3. In this case, the N pixel elements that make up one pixel are, for example, the R (red) element, the G (green) element, and the B (blue) element. In this case, the three pixel element values ​​included in the pixel value of one pixel are the R value, the G value, and the B value. The R value, G value, and B value represent their respective brightness values. Therefore, the difference calculation unit 14 calculates the difference in R values, the difference in G values, and the difference in B values ​​between corresponding pixels in the first two-dimensional image 110 and the second two-dimensional image 120.

[0072] For example, the difference calculation unit 14 may calculate the difference by subtracting the second two-dimensional image 120 from the first two-dimensional image 110, or by subtracting the first two-dimensional image 110 from the second two-dimensional image 120.

[0073] Specifically, for example, the difference calculation unit 14 may calculate the difference by subtracting the corresponding pixel element value of the second two-dimensional image 120 from the pixel element value of the first two-dimensional image 110, or by subtracting the corresponding pixel element value of the first two-dimensional image 110 from the pixel element value of the second two-dimensional image 120.

[0074] Furthermore, the difference calculation unit 14 may perform the following processing on the difference data 130 to generate new difference data (hereinafter referred to as "difference data 130A").

[0075] Specifically, the difference calculation unit 14 sets a first predetermined value as a new difference corresponding to each pixel if the absolute value of the difference between at least one pixel element value is greater than or equal to the threshold TH and the difference is positive. The difference calculation unit 14 also sets a second predetermined value as a new difference corresponding to each pixel if the absolute value of the difference between at least one pixel element value is greater than or equal to the threshold TH and the difference is negative. The difference calculation unit 14 also sets a third predetermined value as a new difference corresponding to each pixel if the absolute value of the differences between N pixel element values ​​is less than the threshold TH.

[0076] The first predetermined value, the second predetermined value, and the third predetermined value are all different from each other. As a result, the difference data 130A is a collection of newly obtained differences (difference values) for each pixel. The storage unit 50 stores the difference data 130A. For example, as shown in Figure 5(c), the difference data 130A is composed of difference data 1120A having the first predetermined value, difference data 1220A having the second predetermined value, and difference data 1000A having the third predetermined value.

[0077] As described above with reference to Figure 5, according to this embodiment, instead of directly analyzing the three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522) to calculate the difference, the difference between two-dimensional images (first two-dimensional image 110 and second two-dimensional image 120) is calculated. Therefore, the processing load on server 1 when calculating the difference can be reduced.

[0078] As an example, if the entire area of ​​the three-dimensional data (point cloud data) constituting a three-dimensional model is analyzed using an octree algorithm and the difference is calculated, the processing load on the computer is large. In contrast, in this embodiment, by calculating the difference between two-dimensional images (first two-dimensional image 110 and second two-dimensional image 120), the processing load on the server 1 is reduced, while indirectly calculating the difference between the three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522).

[0079] In particular, in this embodiment, as shown in Figure 3, the preprocessing for calculating the difference consists of generating a first two-dimensional image 110 representing the first three-dimensional model 521 on a two-dimensional plane using a first viewpoint 61, and generating a second two-dimensional image 120 representing the second three-dimensional model 522 on a two-dimensional plane using the same second viewpoint 81 as the first viewpoint 61. By performing this simple preprocessing, the first two-dimensional image 110 and the second two-dimensional image 120 to be differed can be easily calculated.

[0080] Furthermore, in this embodiment, the image adjustment unit 13 in Figure 2 may perform adjustment processing on at least one of the first two-dimensional image 110 and the second two-dimensional image 120. In this case, the difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120 after performing the adjustment processing.

[0081] The adjustment process includes at least one of the first adjustment process and the second adjustment process.

[0082] The first adjustment process is a process of adjusting the color tone of at least one of the first two-dimensional image 110 and the second two-dimensional image 120. The color tone is the brightness and / or contrast of the image. The image adjustment unit 13, for example, brings the color tone of one of the first two-dimensional image 110 and the second two-dimensional image 120 closer to the color tone of the other image, or matches it to the color tone of the other image.

[0083] For example, if the first time zone T1 is noon and the second time zone T2 is evening, the difference may be large across all pixels. In this case, even if the object has not changed between the first time zone T1 and the second time zone T2, the large difference may lead to the object being mistakenly judged as having changed. Therefore, by adjusting or matching the color tones of the first two-dimensional image 110 and the second two-dimensional image 120 before calculating the difference, changes in the object can be detected with high accuracy.

[0084] The second adjustment process is a process of excluding at least one pixel element from the N pixel elements that make up each pixel of the first two-dimensional image 110 and the second two-dimensional image 120 from the difference calculation. The image adjustment unit 13, for example, excludes from the difference calculation unit 14 a pixel element that depends on the surrounding environment of the object, among the N pixel elements that make up each pixel.

[0085] For example, in the evening, the B value may be larger than the R value and G value compared to noon. In this case, even if the object has not changed between the first time period T1 and the second time period T2, the large difference in the B value may lead to the object being judged as having changed. Therefore, for example, if the first time period T1 is noon and the second time period T2 is evening, the change in the object can be detected with high accuracy by excluding the B element from the R, G, and B elements of each pixel from the difference calculation.

[0086] As described above, by providing the image adjustment unit 13, even if the surrounding environment of the object at the first time T1 when the object corresponding to the first three-dimensional model 521 was measured is different from the surrounding environment of the object at the second time T2 when the object corresponding to the second three-dimensional model 522 was measured, the difference between the first two-dimensional image 110 and the second two-dimensional image 120 can be accurately detected.

[0087] Note that the first and second adjustment processes are merely examples, and any adjustment process may be performed on the first two-dimensional image 110 and the second two-dimensional image 120.

[0088] Furthermore, in this embodiment, the image enhancement unit 15 in Figure 2 may enhance the areas in the two-dimensional image or three-dimensional model where the first two-dimensional image 110 and the second two-dimensional image 120 differ, based on the difference between the two-dimensional image 110 and the second two-dimensional image 120. In this case, when a user views the two-dimensional image or three-dimensional model, they can easily identify the differing areas. For example, the image enhancement unit 15 adds a specific color to the areas in the two-dimensional image or three-dimensional model where the first two-dimensional image 110 and the second two-dimensional image 120 differ, based on the difference between the two-dimensional image 110 and the second two-dimensional image 120.

[0089] Figure 6(a) shows an example of a two-dimensional image 140 that includes regions 1221 and 1121 that differ between the first two-dimensional image 110 and the second two-dimensional image 120. As shown in Figure 6(a), the image enhancement unit 15 enhances regions 1121 and 1221 in the two-dimensional image 140 for regions 141 that do not differ between the first two-dimensional image 110 and the second two-dimensional image 120, based on the difference data 130 or difference data 130A (Figure 5(c)). Region 1121 is identified based on the difference data 1120 or difference data 1120A (Figure 5(c)). Region 1221 is identified based on the difference data 1220 or difference data 1220A.

[0090] For example, region 1121 is highlighted by adding a first specific color 1122 (hatching with diagonal lines extending to the upper left). For example, region 1221 is highlighted by adding a second specific color 1222 (hatching with diagonal lines extending to the upper right).

[0091] The first specific color 1122 and the second specific color 1222 are different. Therefore, the user can easily distinguish between the region 1121 that exists in the first two-dimensional image 110 but not in the second two-dimensional image 120, and the region 1221 that does not exist in the first two-dimensional image 110 but does exist in the second two-dimensional image 120.

[0092] As an example, the image enhancement unit 15 uses the first two-dimensional image 110 or the second two-dimensional image 120 as a base image and generates the two-dimensional image 140 in Figure 6(a) based on the difference data 130 or difference data 130A in Figure 5(c). In this case, for example, the image enhancement unit 15 adds a first specific color 1122 to the regions identified by the difference data 1120 and 1120A in the first two-dimensional image 110 (Figure 5(a)) or the second two-dimensional image 120 (Figure 5(b)), and adds a second specific color 1222 to the regions identified by the difference data 1220 and 1220A. As a result, in the two-dimensional image 140, the regions 1121 and 1221 that differ between the first two-dimensional image 110 and the second two-dimensional image 120 are enhanced.

[0093] The memory unit 50 in Figure 2 stores the two-dimensional image 140. The display control unit 16 then displays the two-dimensional image 140 on the terminal 2 via the communication unit 40. As a result, the user of the terminal 2 can easily recognize the regions 1121 and 1221 where the first two-dimensional image 110 and the second two-dimensional image 120 differ.

[0094] Figure 6(b) shows an example of a three-dimensional model 140A that includes regions 1221A and 1121A where the first two-dimensional image 110 and the second two-dimensional image 120 differ. As shown in Figure 6(b), the three-dimensional model 140A is placed in a virtual space VS where a three-dimensional coordinate system CS is set. Based on the difference data 130 or difference data 130A (Figure 5(c)), the image enhancement unit 15 enhances regions 1121A and 1221A in the three-dimensional model 140A where the first two-dimensional image 110 and the second two-dimensional image 120 do not differ. Region 1121A is identified based on the difference data 1120 or difference data 1120A (Figure 5(c)) and is enhanced by adding a first specific color 1122. Region 1221A is identified based on differential data 1220 or differential data 1220A and is highlighted by the addition of a second specific color 1222.

[0095] As an example, the image enhancement unit 15 uses either the first three-dimensional model 521 corresponding to the first two-dimensional image 110 or the second three-dimensional model 522 corresponding to the second two-dimensional image 120 as a base model, and generates the three-dimensional model 140A in Figure 6(b) based on the difference data 130 or difference data 130A in Figure 5(c). In this case, the first three-dimensional model 521 and the second three-dimensional model 522 are pre-assigned surfaces, and material information is attached to these surfaces.

[0096] For example, the image enhancement unit 15 adds a first specific color 1122 in place of material information to the regions identified by the difference data 1120 and 1120A in the first three-dimensional model 521 or the second three-dimensional model 522. Furthermore, the image enhancement unit 15 adds a second specific color 1222 in place of material information to the regions identified by the difference data 1220 and 1220A in the first three-dimensional model 521 or the second three-dimensional model 522. As a result, in the three-dimensional model 140A, the regions 1121A and 1221A where the first two-dimensional image 110 and the second two-dimensional image 120 differ are enhanced.

[0097] The memory unit 50 in Figure 2 stores the three-dimensional model 140A. The display control unit 16 then displays the three-dimensional model 140A on the terminal 2 via the communication unit 40. As a result, the user of the terminal 2 can easily recognize the regions 1121A and 1221A on the three-dimensional model 140A where the first two-dimensional image 110 and the second two-dimensional image 120 differ.

[0098] Here, the image generation unit 12 will be described with reference to Figures 2 and 3. The image generation unit 12 may set up multiple different first viewpoints 61 in the first virtual space VS1 (Figure 3(a)). The image generation unit 12 may then generate a first two-dimensional image 110 for each of the multiple different first viewpoints 61. Furthermore, the image generation unit 12 may set up multiple different second viewpoints 81 in the second virtual space VS2 (Figure 3(b)). The image generation unit 12 may then generate a second two-dimensional image 120 for each of the multiple different second viewpoints 81. Furthermore, the difference calculation unit 14 may calculate multiple differences corresponding to the multiple first viewpoints 61 and the multiple second viewpoints 81. Hereinafter, a first viewpoint 61 and a second viewpoint 81 identical to the first viewpoint 61 will be referred to as a "viewpoint pair". In this case, a first two-dimensional image 110 and a second two-dimensional image 120 are generated for each different viewpoint pair. Therefore, for each different viewpoint pair, a difference is generated between the first two-dimensional image 110 and the second two-dimensional image 120. In other words, multiple differences are generated for each of the multiple viewpoint pairs. Thus, the difference between the object at the first time T1 and the object at the second time T2 can be detected with high accuracy across the entire area of ​​the object.

[0099] For example, in the first virtual space VS1 shown in Figure 3(a), the image generation unit 12 sets multiple first viewpoints 61 to multiple points selected from vertices a1 to a8, midpoints b1 to b12, and centers c1 to c6. Then, in the second virtual space VS2 shown in Figure 3(b), the image generation unit 12 sets multiple second viewpoints 81, each identical to the multiple first viewpoints 61.

[0100] Next, an image analysis method according to this embodiment will be described with reference to Figures 2 and 7. Figure 7 is a flowchart of the image analysis method. The image analysis method is executed by Server 1. As shown in Figure 7, the image analysis method includes steps S1 to S12. The computer program stored in the storage unit 50 causes the arithmetic unit 10 to execute steps S1 to S12. In other words, the computer program product realizes steps S1 to S12 when the computer program is executed by the arithmetic unit 10. The arithmetic unit 10 corresponds to an example of a "computer" in this disclosure.

[0101] As shown in Figures 2 and 7, first, in step S1, the model generation unit 11 acquires a first three-dimensional model 521. Specifically, the model generation unit 11 generates a first three-dimensional model 521 representing the object at first time T1 based on multiple two-dimensional images generated by imaging the object from multiple different imaging positions at first time T1. Alternatively, the model generation unit 11 sets three-dimensional data 513 based on the measurement results of the object at first time T1 into the first three-dimensional model 521. Then, the model generation unit 11 sets a first rectangular parallelepiped 71 surrounding the first three-dimensional model 521.

[0102] Next, in step S2, the image generation unit 12 generates a first two-dimensional image 110 in which the first three-dimensional model 521 is represented on a two-dimensional plane by a first viewpoint 61 in the first virtual space VS1. The first viewpoint 61 is set on the first rectangular parallelepiped 71.

[0103] Next, in step S3, the model generation unit 11 acquires a second three-dimensional model 522. Specifically, the model generation unit 11 generates a second three-dimensional model 522 representing the object in the second time T2 based on multiple two-dimensional images generated by imaging the object from multiple different imaging positions in the second time T2. Alternatively, the model generation unit 11 sets three-dimensional data 513 based on the measurement results of the object in the second time T2 into the second three-dimensional model 522. Then, the model generation unit 11 sets a second rectangular parallelepiped 72 surrounding the second three-dimensional model 522. In this disclosure, setting the second rectangular parallelepiped 72 is not mandatory.

[0104] Next, in step S4, the image generation unit 12 generates a second two-dimensional image 120 in which the second three-dimensional model 522 is represented on a two-dimensional plane by the second viewpoint 81 of the second virtual space VS2. In the three-dimensional coordinate system CS set in the first virtual space VS1 and the second virtual space VS2, the position and line of sight direction 82 of the second viewpoint 81 are the same as the position and line of sight direction 82 of the first viewpoint 61, respectively.

[0105] Next, in step S5, the image adjustment unit 13 performs an adjustment process on at least one of the first two-dimensional image 110 and the second two-dimensional image 120.

[0106] Next, in step S6, the difference calculation unit 14 calculates the difference between the first two-dimensional image 110 and the second two-dimensional image 120 after the adjustment process has been executed, and generates difference data 130.

[0107] Next, in step S7, the image enhancement unit 15 enhances the difference region between the first two-dimensional image 110 and the second two-dimensional image 120 in the two-dimensional image 140 or the three-dimensional model 140A based on the difference indicated by the difference data 130.

[0108] Next, in step S8, the image generation unit 12 determines whether or not it has generated difference data 130 indicating the difference for all viewpoint pairs (first viewpoint 61 and second viewpoint 81).

[0109] If a negative result (NO) is obtained in step S8, the process proceeds to step S9.

[0110] Next, in step S9, the image generation unit 12 changes the viewpoint pair (first viewpoint 61 and second viewpoint 81).

[0111] Next, in step S10, the image generation unit 12 generates a first two-dimensional image 110 in which the first three-dimensional model 521 is represented on a two-dimensional plane with the modified first viewpoint 61.

[0112] Next, in step S11, the image generation unit 12 generates a second two-dimensional image 120, which represents the second three-dimensional model 522 on a two-dimensional plane using the modified second viewpoint 81. Then, the process proceeds to step S5. Steps S5 to S11 are repeated until difference data 130 is calculated for all viewpoint pairs (first viewpoint 61 and second viewpoint 81).

[0113] On the other hand, if a positive result is obtained in step S8 (YES), the process proceeds to step S12.

[0114] Next, in step S12, the display control unit 16, in response to a request from the user's terminal 2, displays a two-dimensional image 140 or a three-dimensional model 140A on the terminal 2 via the communication unit 40, in which the difference region between the first two-dimensional image 110 and the second two-dimensional image 120 is highlighted. The image analysis method then ends.

[0115] As explained above with reference to Figure 7, the image analysis method according to this embodiment calculates the difference between two-dimensional images (first two-dimensional image 110 and second two-dimensional image 120) rather than directly analyzing three-dimensional models (first three-dimensional model 521 and second three-dimensional model 522). Therefore, the processing load on server 1 when calculating the difference can be reduced.

[0116] Preferred embodiments and modifications of the present disclosure have been described in detail above with reference to the attached drawings, but the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations can be conceived within the scope of the technical idea set forth in the claims, and these too are understood to fall within the technical scope of the present disclosure.

[0117] The apparatus or system described herein may be implemented as a single apparatus, or it may be implemented by a plurality of apparatuses (e.g., a cloud server) that are partially or completely connected by a network. For example, some or all of the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may be implemented by the same computer or server. For example, the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may each be implemented by separate computers or servers. Alternatively, for example, the model generation unit 11, image generation unit 12, image adjustment unit 13, difference calculation unit 14, image enhancement unit 15, and display control unit 16 may be implemented by a terminal 2 or a three-dimensional measuring device 7. For example, video data 511, still image dataset 512, three-dimensional data 513, first three-dimensional model 521, second three-dimensional model 522, first two-dimensional image 110, second two-dimensional image 120, difference data 130, 130A, two-dimensional image 140, and three-dimensional model 140A may each be stored in separate storage devices or servers.

[0118] The series of processes performed by the apparatus described herein may be implemented using software, hardware, or a combination of software and hardware. Computer programs for implementing each function of the arithmetic unit 10 according to this embodiment can be created and implemented on a PC or the like. Furthermore, a computer-readable recording medium containing such a computer program can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, and flash memory. Alternatively, the computer program may be distributed without using a recording medium, for example, via a network.

[0119] Furthermore, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0120] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0121] Furthermore, the following configurations also fall within the technical scope of this disclosure.

[0122] (Item 1) An image generation unit that generates a first two-dimensional image and a second two-dimensional image, The system includes a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image, The image generation unit, A first two-dimensional image is generated, which represents a first three-dimensional model, created based on the measurement results of the shape of an object in real space at time 1, on a two-dimensional plane from a first viewpoint in a first virtual space. A second two-dimensional image is generated, which represents the second three-dimensional model, generated in the real space based on the measurement results of the shape of the object at the second time, on a two-dimensional plane from a second viewpoint in the second virtual space. The same coordinate system is set for the first virtual space and the second virtual space. An image analysis device in which, in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

[0123] (Item 2) The first viewpoint is set on a rectangular parallelepiped surrounding the first three-dimensional model, and is the image analysis device described in item 1.

[0124] (Item 3) The first viewpoint is set at a vertex, midpoint of an edge, or center of a face of the rectangular parallelepiped, as described in item 2 of the image analysis device.

[0125] (Item 4) The image analysis device described in item 2, wherein the first viewpoint is set at a vertex of the rectangular parallelepiped and faces the direction of the diagonal line extending from the vertex.

[0126] (Item 5) The image generation unit, In the aforementioned first virtual space, set up multiple different aforementioned first viewpoints, For each of the aforementioned multiple different first viewpoints, the first two-dimensional image is generated. In the aforementioned second virtual space, set up multiple different aforementioned second viewpoints. For each of the aforementioned multiple different second viewpoints, the second two-dimensional image is generated. The image analysis device described in any of items 1 to 4, wherein the difference calculation unit calculates a plurality of differences corresponding to the plurality of first viewpoints and the plurality of second viewpoints.

[0127] (Item 6) The system further includes an image adjustment unit that performs adjustment processing on at least one of the first two-dimensional image and the second two-dimensional image, The adjustment process includes at least one of the first adjustment process and the second adjustment process, The first adjustment process is a process to adjust the color tone of at least one of the images, The second adjustment process is a process of excluding at least one pixel element from the set of pixels of the first two-dimensional image and the second two-dimensional image from the set of differences. The image analysis device described in any of items 1 to 5, wherein the difference calculation unit calculates the difference between the first two-dimensional image and the second two-dimensional image after the adjustment process has been performed.

[0128] (Item 7) The image analysis device according to any one of items 1 to 6, further comprising an image enhancement unit that enhances the region in the two-dimensional image or three-dimensional model where the first two-dimensional image and the second two-dimensional image differ based on the aforementioned difference.

[0129] (Item 8) The process involves generating a first two-dimensional image by representing a first three-dimensional model, which is generated in real space based on the measurement results of the object's shape at time 1, on a two-dimensional plane from a first viewpoint in a first virtual space, and The steps include generating a second two-dimensional image in which a second three-dimensional model, generated based on the measurement results of the shape of the object in the real space at a second time, is represented on a two-dimensional plane from a second viewpoint in a second virtual space, and The steps include calculating the difference between the first two-dimensional image and the second two-dimensional image, The same coordinate system is set for the first virtual space and the second virtual space. An image analysis method in which, in the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

[0130] (Item 9) On the computer, The process involves generating a first two-dimensional image by representing a first three-dimensional model, which is generated in real space based on the measurement results of the object's shape at time 1, on a two-dimensional plane from a first viewpoint in a first virtual space, and The steps include generating a second two-dimensional image in which a second three-dimensional model, generated based on the measurement results of the shape of the object in the real space at a second time, is represented on a two-dimensional plane from a second viewpoint in a second virtual space, and The process involves performing the steps of calculating the difference between the first two-dimensional image and the second two-dimensional image, The same three-dimensional coordinate system is set in the first virtual space and the second virtual space. A computer program in which, in the aforementioned three-dimensional coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively. [Industrial applicability]

[0131] This disclosure provides an image analysis device, an image analysis method, and a computer program, and has industrial applicability. [Explanation of Symbols]

[0132] 1 Server (image analysis device), 11 Model generation unit, 12 Image generation unit, 13 Image adjustment unit, 14 Difference calculation unit, 15 Image enhancement unit, 16 Display control unit

Claims

1. An image generation unit that generates a first two-dimensional image and a second two-dimensional image, The system includes a difference calculation unit that calculates the difference between the first two-dimensional image and the second two-dimensional image, The image generation unit, A first two-dimensional image is generated, which represents a first three-dimensional model, generated in real space based on the measurement results of the shape of the object at the first time, on a two-dimensional plane from a first viewpoint in a first virtual space. A second two-dimensional image is generated, which represents the second three-dimensional model, generated in the real space based on the measurement results of the shape of the object at the second time, on a two-dimensional plane from a second viewpoint in the second virtual space. The same coordinate system is set for the first virtual space and the second virtual space. An image analysis device in which, in the coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

2. The image analysis apparatus according to claim 1, wherein the first viewpoint is set on a rectangular parallelepiped surrounding the first three-dimensional model.

3. The image analysis apparatus according to claim 2, wherein the first viewpoint is set at a vertex, midpoint of an edge, or center of a face of the rectangular parallelepiped.

4. The image analysis apparatus according to claim 2, wherein the first viewpoint is set at a vertex of the rectangular parallelepiped and faces the direction of the diagonal line extending from the vertex.

5. The image generation unit, In the first virtual space, set up a plurality of different first viewpoints, For each of the aforementioned multiple different first viewpoints, the first two-dimensional image is generated. In the aforementioned second virtual space, set up multiple different aforementioned second viewpoints, For each of the aforementioned multiple different second viewpoints, the second two-dimensional image is generated. The image analysis apparatus according to claim 1 or claim 2, wherein the difference calculation unit calculates a plurality of differences corresponding to the plurality of first viewpoints and the plurality of second viewpoints.

6. The system further includes an image adjustment unit that performs adjustment processing on at least one of the first two-dimensional image and the second two-dimensional image, The adjustment process includes at least one of the first adjustment process and the second adjustment process, The first adjustment process is a process to adjust the color tone of at least one of the images, The second adjustment process is a process of excluding at least one pixel element from the plurality of pixel elements that constitute each pixel of the first two-dimensional image and the second two-dimensional image from the difference target. The image analysis apparatus according to claim 1 or 2, wherein the difference calculation unit calculates the difference between the first two-dimensional image and the second two-dimensional image after the adjustment process has been performed.

7. The image analysis apparatus according to claim 1 or claim 2, further comprising an image enhancement unit that enhances the region in the two-dimensional image or three-dimensional model where the first two-dimensional image and the second two-dimensional image differ based on the aforementioned difference.

8. The process involves generating a first two-dimensional image by representing a first three-dimensional model, which is generated in real space based on the measurement results of the shape of an object at time 1, on a two-dimensional plane from a first viewpoint in a first virtual space, and The steps include generating a second two-dimensional image in which a second three-dimensional model, generated based on the measurement results of the shape of the object in the real space at a second time, is represented on a two-dimensional plane from a second viewpoint in a second virtual space, The step includes calculating the difference between the first two-dimensional image and the second two-dimensional image, The same coordinate system is set for the first virtual space and the second virtual space. An image analysis method in which, in the coordinate system, the position and line of sight direction of the second viewpoint are the same as the position and line of sight direction of the first viewpoint, respectively.

9. On the computer, The process involves generating a first two-dimensional image by representing a first three-dimensional model, which is generated in real space based on the measurement results of the shape of an object at time 1, on a two-dimensional plane from a first viewpoint in a first virtual space, and The steps include generating a second two-dimensional image in which a second three-dimensional model, generated based on the measurement results of the shape of the object in the real space at a second time, is represented on a two-dimensional plane from a second viewpoint in a second virtual space, The process involves performing the steps of calculating the difference between the first two-dimensional image and the second two-dimensional image, The same three-dimensional coordinate system is set in the first virtual space and the second virtual space. A computer program in which, in the aforementioned three-dimensional coordinate system, the position and line of sight of the second viewpoint are the same as the position and line of sight of the first viewpoint, respectively.

Images