Image processing device, method and program for controlling image processing device

Abstract

To present a screen that makes it easier to check changes over time in abnormalities detected in images of the same object taken at different times.SOLUTION: Deformations of an object are detected and acquired from a first image of the object and a second image of the object captured at a time different from the first image, and differential deformations and common deformations are extracted as changes over time from the detected deformations of the object. Then, when displaying the first image or the second image, the display modes of the differential deformations and common deformations are made different between the first image and the second image.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a control method of the image processing apparatus, and a program.

Background Art

[0002] In recent years, instead of manually inspecting and recording changes such as damage (cracks, efflorescence) of structures such as bridges, a method of detecting changes in structures in an image by image recognition technology and recording information such as the type, size, and position of the changes has become widespread. Further, changes are detected from images taken at a plurality of different times for the same object, and the temporal change of the changes in the structure is also displayed by extracting the difference between the change detection results. Patent Document 1 discloses a technique for creating a damage diagram that represents the extracted damage and the extraction result of the recorded past damage in different manners or represents the difference from the extraction result of the recorded past damage in different manners so as to grasp the temporal change of the damage.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when confirming the temporal change of the change, the change may be confirmed in more detail. In the method described in Patent Document 1, although the temporal change can be displayed among a plurality of detected changes, there is a problem that it is difficult to confirm when it is desired to confirm the change in more detail.

[0005] In view of the above problems, an object of the present invention is to be able to present a screen that is easier to confirm the temporal change of the changes respectively detected from images taken at a plurality of different times for the same object.

Means for Solving the Problem

[0006] An image processing apparatus according to the present invention includes an acquisition unit that acquires a change over time in the deformation of an object between a first image obtained by photographing the object and a second image obtained by photographing the object at a time different from the first image, and a display control unit that displays the first image or the second image on a display device so as to display the deformation in a display mode corresponding to the change over time in the deformation acquired by the acquisition unit. The display control unit is characterized in that the display mode of the deformation is made different between the first image and the second image.

Advantages of the Invention

[0007] According to the present invention, it is possible to present a screen that is easier to confirm with respect to the change over time in the deformation respectively detected from images taken of the same object at a plurality of different times.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Hereinafter, an example of detecting cracks as deformation from an image of a structure such as a bridge will be described, but the object of photography and the type of deformation are not limited to these. For example, the present invention may be applied when detecting deformation such as a tumor in a human body in a medical image.

[0010] (First Embodiment) FIG. 1 is a block diagram showing a functional configuration example of an image processing apparatus 100 in the present embodiment. The image processing apparatus 100 includes an image acquisition unit 101 that acquires an image, a deformation acquisition unit 102 that acquires deformation of the image, a change-over-time acquisition unit 103 that acquires a change-over-time of the deformation, and a display control unit 104 that determines deformation to be superimposed on the image and its mode and displays it together with the image. Details of each process by these configurations will be described later.

[0011] FIG. 2 is a block diagram showing an example of the hardware configuration of the image processing apparatus 100 according to the present embodiment. In FIG. 2, a CPU (Central Processing Unit) 201 executes a control program of the image processing apparatus 100. A ROM 202 stores a control program executed by the CPU 201 and the like. A RAM 203 is used to provide a work area for the CPU 201. A network IF 204 is an interface for communicating with an application outside the apparatus. A storage device 205 is a storage device such as a hard disk and stores various data. A display device 206 displays the processing result of the CPU 201, various data, and the like. An operation I / F 207 is an interface for inputting operation information from operation members such as a mouse and a keyboard. A bus 208 is a bus for connecting these components.

[0012] Subsequently, a specific processing procedure for detecting a change and displaying an image indicating the change by the image processing apparatus 100 according to the present embodiment will be described. Hereinafter, with reference to the flowchart of FIG. 3, an example will be described in which an image of a structure such as a bridge taken three years ago is obtained as a first image, and an image of the same object taken recently is obtained as a second image, and cracks are detected as a change from these images. Also, it is assumed that the first image and the second image are stored in advance in the storage device 205.

[0013] FIG. 3 is a flowchart showing an example of a processing procedure for displaying a change over time of a change by the image processing apparatus 100 in the present embodiment. First, in step S301, an image acquisition unit 101 acquires a first image from the storage device 205.

[0014] Next, in step S302, the deformation acquisition unit 102 acquires the deformation of the first image acquired in step S301 (hereinafter referred to as the first deformation). In the present embodiment, the deformation acquisition unit 102 performs image recognition processing on the first image and detects the deformation of the object shown in the image. Note that the method for acquiring the deformation is not limited to this. For example, if a deformation has been detected from the first image in the past and the information is stored in the storage device 205, the deformation information corresponding to the first image may be acquired from the storage device 205. Alternatively, the user may operate the operation member while viewing the first image and manually specify the position of the deformation to acquire the first deformation.

[0015] Next, in step S303, the image acquisition unit 101 acquires a second image from the storage device 205. Then, in step S304, the deformation acquisition unit 102 acquires the deformation of the second image acquired in step S303 (the second deformation). Note that the method for acquiring the deformation is the same as that in step S302.

[0016] FIG. 4 is a diagram for explaining the deformation acquired by the deformation acquisition unit 102. FIG. 4(a) is a diagram showing an example of the first image acquired in step S301, and FIG. 4(b) is a diagram showing an example of the first deformation detected from the first image. Similarly, FIG. 4(c) is a diagram showing an example of the second image acquired in step S303, and FIG. 4(d) is a diagram showing an example of the second deformation detected from the second image. The first image and the second image are taken at different times, and during this period, deformations such as cracks may progress, or the deformation may disappear due to repair work. Also, there may be a difference in image quality depending on the imaging equipment, imaging position, and imaging time. In that case, even if the actual deformation of the object has not changed, there may be a difference in the deformations detected as the first deformation and the second deformation.

[0017] In the examples shown in FIGS. 4(b) and 4(d), the first deformation and the second deformation are each displayed as an image. However, in steps S302 and S304, the deformation acquisition unit 102 acquires the deformations as deformation data in a table format. FIG. 5 is a diagram showing an example of the deformation data. As shown in FIG. 5, the deformation data is composed of items of ID, legend, maximum width, number of points, and point coordinates.

[0018] "ID" is a deformation ID representing a specific deformation assigned to each deformation. "Legend" represents the classification of the deformation. In the example of FIG. 5, the classification is defined by the degree of crack width, and the classification is assigned to each deformation. "Maximum width" represents the maximum crack width within one deformation, and "number of points" is the number of points for representing one deformation by one or more line segments. Also, "point coordinates" represent the coordinates of each point on the image. For example, in the case of a deformation with a "number of points" of "3", the point coordinates will have three, and the deformation will be represented by two line segments connecting the three points in order. When the deformation is drawn based on the deformation data as shown in FIG. 5, the deformations as shown in FIGS. 4(b) and 4(d) will be obtained. Note that the data format of the deformation data acquired by the deformation acquisition unit 102 is not limited to the format shown in FIG. 5, and any data format can be used as long as it represents the deformation.

[0019] Next, in step S305, the change-over-time acquisition unit 103 extracts and acquires the change-over-time of the deformation from the first deformation and the second deformation. Here, the change-over-time is information representing the differential deformation and the common deformation as the difference between the first deformation and the second deformation with different shooting times.

[0020] FIG. 6 is a diagram for explaining the change over time acquired by the change-over-time acquisition unit 103. FIG. 6(a) is a diagram showing an example of a first change state acquired from the first image, and FIG. 6(b) is a diagram showing an example of a second change state acquired from the second image. On the other hand, FIG. 6(c) is a diagram showing an example of a differential change state determined to be included in the first change state but not in the second change state by the change-over-time extraction process by the change-over-time acquisition unit 103. Further, FIG. 6(d) is a diagram showing an example of a differential change state determined to be included in the second change state but not in the first change state by the same change-over-time extraction process. Furthermore, FIG. 6(e) is a diagram showing an example of a common change state included in the first change state, which is determined to be common between the first change state and the second change state by the change-over-time extraction process. Also, FIG. 6(f) is a diagram showing an example of a common change state included in the second change state, which is determined to be common between the first change state and the second change state by the change-over-time extraction process. Note that when comparing the example of FIG. 6(e) and the example of FIG. 6(f), although they are the same common change state, since they are extracted from the first change state and the second change state respectively, the coordinates and shapes are slightly different.

[0021] In the change-over-time extraction process by the change-over-time acquisition unit 103, differential change states and common change states as shown in FIGS. 6(c) to 6(f) are extracted for the first change state and the second change state respectively. Specifically, the change-over-time acquisition unit 103 determines which change state is a difference and which is common from the change state data as shown in FIG. 5. And when there is a partial commonality and a difference for one common change state, the change state ID is divided into a differential change state and a common change state, and information such as point coordinates is retained respectively. Also, for the common change state, information on the correspondence relationship of which ID of the common change state of the first change state corresponds to which ID of the common change state of the second change state is also retained.

[0022] Next, in step S306, the display control unit 104 displays either the first image or the second image on the display device 206. Then in step S307, the display control unit 104 determines the mode of the change over time to be displayed from the image displayed in step S306, and superimposes and displays the change over time on the displayed image.

[0023] Furthermore, in step S308, the display control unit 104 determines whether or not it has received an operation to switch the display from the currently displayed image to the other image (the second image when the first image is being displayed) via the operation I / F 207. If the display control unit 104 determines that it has received an operation to switch the display to the other image, the process returns to step S306, and the display control unit 104 displays the other image. On the other hand, if the display control unit 104 determines that it has not received an operation to switch the display to the other image, the process proceeds to step S309. Then, in step S309, the display control unit 104 determines whether or not it has received an operation to end the display of the image via the operation I / F 207. If the display control unit 104 determines that it has not received an operation to end the display of the image via the operation I / F 207, the process returns to step S308. On the other hand, if the display control unit 104 determines that it has received an operation to end the display of the image via the operation I / F 207, the process ends as it is.

[0024] FIG. 7 is a diagram showing an example of the image displayed in step S307. Here, FIG. 7(a) represents an image in which a change is displayed in a predetermined manner in the first image, and FIG. 7(b) represents an image in which a change is displayed in a predetermined manner in the second image.

[0025] In the example of FIG. 7(a), based on the result of the change over time, the change included in the first image is displayed in a predetermined manner (thick solid line), and the differential change extracted from the second image and not included in the first image is displayed in another manner (thin dotted line). In FIG. 7(a), the change 701 is a differential change extracted from the first image but not from the second image. The changes 702 and 703 are common changes extracted from the first image and also from the second image. Note that the change 703 is a common change similar to the change 702, but is a common change with different crack widths, etc. between the first change and the second change. The change 704 is a differential change extracted from the second image but not from the first image.

[0026] As shown in Fig. 7(a), when displaying the first image, the deformations 701 to 703 are the deformations included in the first image, so they are displayed with thick solid lines. On the other hand, the deformation 704 not included in the first image is displayed with thin dotted lines so that it can be seen that it is not included in the first image.

[0027] Note that the deformations 701 to 704 may be further colored according to the deformation so that they can be distinguished as differential deformations or common deformations. Also, in the example of Fig. 7(a), among the common deformations, the deformations extracted from the second image are difficult to distinguish from the deformations 702 and 703, so they are not displayed.

[0028] On the other hand, similarly in the example of Fig. 7(b), based on the result of the change over time, the deformations included in the second image are displayed in a predetermined manner (thick solid line), and the differential deformations extracted from the first image and not included in the second image are displayed in another manner (thin dotted line). The deformation 705 is a differential deformation extracted from the first image but not from the second image. The deformations 706 and 707 are common deformations extracted from the second image and also from the first image. Note that the deformation 707 is a common deformation with different crack widths and the like as described above. The deformation 708 is a differential deformation extracted from the second image but not from the first image.

[0029] Similarly in the case of Fig. 7(b), the deformations 706 to 708 included in the second image are displayed with thick solid lines, and the deformation 705 not included in the second image is displayed with thin dotted lines. Also, as in Fig. 7(a), the color may be changed according to the deformation. Also, in the example of Fig. 7(b), among the common deformations, the deformations extracted from the first image are not displayed.

[0030] In this embodiment, as an operation for switching the display, a simple operation such as clicking a mouse button by the user can be considered. The display is switched by a simple operation such as clicking on the first image or the second image on one screen, and the user can separately check the difference between the deformation at that time and the deformation of the other image while checking the state of the object in the first image and the state of the object in the second image.

[0031] On the other hand, for example, the first image and the second image may be arranged and displayed. In this case, the process of step S308 in FIG. 3 is omitted, and in step S306, the display control unit 104 displays both the first image and the second image on the display device 206. Then, in step S307, in each of the first image and the second image, the display mode may be changed and displayed according to the differential change and the common change of the change over time. Also, as variations of the display mode of the change, any combination of modes such as the color of the line, the thickness, the dashed line, and the blinking may be used.

[0032] FIG. 8 is a diagram showing an example in which the display control unit 104 emphasizes the difference over time and superimposes the change on the image. FIG. 8(a) shows an example in which the difference over time is emphasized and the change is superimposed on the first image, and FIG. 8(b) shows an example in which the difference over time is emphasized and the change is superimposed on the second image. In FIGS. 8(a) and 8(b), the change 801 is a differential change included only in the first change, and the change 802 is a differential change included only in the second change. The changes 803 and 308 are both common changes included in both the first change and the second change. Note that the change 804 is assumed to be a common change with a different crack width or the like.

[0033] In the example of FIG. 8(a), the differential change 801 is highlighted, and in the example of FIG. 8(b), the differential change 802 is highlighted. Also, since the change 803 is a common change, it is displayed in a mode different from the differential change. However, since the change 804 is a common change with different change sizes or the like in the first image and the second image, it may be highlighted in the same way as the differential change.

[0034] Also, the display form in the present embodiment is not limited to the examples of FIGS. 7 or 8. The respective display modes may be determined so that the differential change, the common change, and the differential change not included in the image of the change over time included in the image can be distinguished depending on whether either the first image or the second image is displayed.

[0035] As described above, according to the present embodiment, switching between the first image and the second image is facilitated, and the temporal change of the deformation is displayed in different display modes for the first image and the second image. Thereby, the user can more detailedly confirm the temporal change of the deformation while checking the first image and the second image.

[0036] (Second Embodiment) In the first embodiment, an example was described in which only the common deformation extracted from the displayed image was displayed and the common deformation extracted from the other image was not displayed because it was difficult to distinguish. On the other hand, both common deformations may be displayed together. In this case, it is conceivable to change the display mode based on whether it is extracted from the displayed image.

[0037] Further, the difference between the common deformation extracted from the displayed image and the common deformation extracted from the other image may be evaluated, and the display of the common deformation may be changed according to the difference. In this case, for example, when the difference such as the crack width is less than a predetermined value, only the common deformation extracted from the displayed image is displayed, and when the difference is greater than or equal to the predetermined value, both common deformations are displayed. Note that, as a method for evaluating the difference, the difference in the shape of the deformation, the crack width, etc. may be evaluated as described above, or the distance between the common deformation extracted from the displayed image and the common deformation extracted from the other image may be evaluated.

[0038] As described above, in the case of a common deformation with a large difference, by displaying both to allow the user to confirm the difference, the user can perform confirmation according to the difference in the deformation.

[0039] (Third Embodiment) In the first and second embodiments, for the first image and the second image in which the same object is photographed at different times, the display mode of each deformation of the temporal change is determined according to the displayed image. On the other hand, the user may be allowed to specify a partial region and switch the display mode of the deformation in the partial region. Hereinafter, only the points different from the first embodiment will be described.

[0040] FIG. 9 is a block diagram showing a functional configuration example of the image processing apparatus 900 in the present embodiment. As shown in FIG. 9, in addition to the configuration shown in FIG. 1, the image processing apparatus 900 according to the present embodiment further includes a specifying unit 901. The specifying unit 901 specifies a position in the displayed image according to an operation of an operation member such as a mouse by the user.

[0041] FIG. 10 is a diagram for explaining a state of displaying a partial area based on the coordinates specified by the specifying unit 901. FIG. 10(a) is a diagram showing a state of switching only the display mode of the partial area based on the position specified by the user. In FIG. 10(a), an image 1001 shows a first image obtained by photographing an object. First, it is assumed that a screen as shown in FIG. 7(a) is displayed on the display device 206 according to the processing procedure of FIG. 3.

[0042] Here, when the user operates a mouse or the like to specify a position, the specifying unit 901 superimposes and displays a partial area 1002 of a predetermined size at the specified position as shown in FIG. 10(a). At this time, only within the partial area 1002, the display is switched from the first image to the second image, and a modified state is displayed according to the display mode when the second image is displayed. For example, a differential modified state not included in the first image is displayed as a thin dotted line in the first image, but as a thick solid line in the second image. Therefore, as shown in FIG. 10(a), only within the partial area 1002, according to the display mode of the second image, only the differential modified state within the partial area 1002 is displayed as a thick solid line.

[0043] FIG. 10(b) is a diagram showing an example of displaying a partial area based on the position indicated by the user on a separate screen. In FIG. 10(b), when the user specifies a position with the cursor 1003 and clicks, the specifying unit 901 displays the partial area 1004 based on the position specified by the cursor 1003 on a separate screen. Also in this case, in the partial area 1004, the differential modified state is displayed in the display mode of the second image.

[0044] As described above, by switching or separately displaying images with different shooting times based on the instructions of the user, the user can make a confirmation according to the difference in the change.

[0045] (The Fourth Embodiment) In the first embodiment described above, an example in which the change acquisition unit 102 detects and acquires a change from an image by image recognition processing, or an example in which the user acquires a change by manually specifying the position of the change while checking the image has been described. On the other hand, the image may be transmitted to another server, and the change may be detected by image recognition processing on that server. Similarly, the change over time may also be acquired by another server. In this case, the first change and the second change, or the first image and the second image are transmitted to another server, and the change over time of the change is extracted on that server, and the information on the change over time is acquired from the server. In this embodiment, an example in which the detection of the change and the extraction of the change over time are performed by an external device such as a server will be described, and only the points different from the first embodiment will be described.

[0046] FIG. 11 is a diagram for explaining the outline of the image processing system 1100 in this embodiment. As shown in FIG. 11, the image processing system according to this embodiment includes an image processing apparatus 1101, a change detection server 1102, and a change over time extraction server 1103.

[0047] The image processing device 1101 is a terminal having the configuration shown in FIGS. 1 and 2, and displays a first image and a second image on the display device 206 of the image processing device 1101. The deformation detection server 1102 is an external device that acquires an image from the image processing device 1101 and detects deformation from the image by image recognition processing. The time-varying change extraction server 1103 is an external device that extracts the time-varying change of the deformation from the first deformation and the second deformation. Note that the deformation detection server 1102 and the time-varying change extraction server 1103 may be the same device. Further, in the image processing system 1100 according to the present embodiment, the first image and the second image may be held by the image processing device 1101 or the deformation detection server 1102. However, holding these images by the image processing device 1101 can display the first image and the second image at a higher speed.

[0048] Further, the deformation detection server 1102 acquires the first image and the second image and detects the first deformation and the second deformation, respectively. However, since the shooting times of the first image and the second image are significantly different, the acquisition times of the deformation are often significantly different. Therefore, when extracting the time-varying change, the detected first deformation may be held by the image processing device 1101, and only the second image may be transmitted to the deformation detection server 1102 to detect the second deformation. Further, when acquiring the deformation from the deformation detection server 1102, deformation data as shown in FIG. 5 is acquired.

[0049] On the other hand, when acquiring the time-varying change, the image processing device 1101 transmits the first deformation and the second deformation to the time-varying change extraction server 1103. Further, when the time-varying change extraction server 1103 obtains the time-varying change using the image information as well, the image processing device 1101 may transmit the first image and the second image to the time-varying change extraction server 1103 together. The information obtained from the time-varying change extraction server 1103 includes a differential deformation included in the first deformation and not included in the second deformation, a common deformation included in the first deformation, a differential deformation included in the second deformation and not included in the first deformation, and a common deformation included in the second deformation.

[0050] Also, in this embodiment, when displaying an image with a modified state superimposed, the image processing apparatus 1101 switches variously the mode of the modified state to be superimposed according to the image to be displayed. However, the superimposed image may be generated by the time-varying extraction server 1103. In this case, the time-varying extraction server 1103 determines the mode of the modified state to be superimposed on the first image and generates the first superimposed image with the modified state superimposed, and determines the mode of the modified state to be superimposed on the second image and generates the second superimposed image with the modified state superimposed, and transmits them to the image processing apparatus 1101.

[0051] (The Fifth Embodiment) In the above-described first embodiment, an example in which images are displayed in the same image coordinate system for the first image and the second image has been described. That is, in the first embodiment, it is premised that the coordinate systems of the first image and the second image are aligned in advance. In this embodiment, an example in which the respective images are displayed while the coordinate systems of the first image and the second image are not aligned will be described. In this case, in order to quickly switch between the first image and the second image and superimpose and display the modified state, it is conceivable to hold the coordinate data of both the coordinate system of the first image and the coordinate system of the second image for each modified state. Hereinafter, only the points different from the first embodiment will be described.

[0052] FIG. 12(a) is a diagram showing an example of modified state data acquired from the first image in this embodiment, and FIG. 12(b) is a diagram showing an example of modified state data acquired from the second image in this embodiment. In FIG. 12(a), the point coordinates of the modified state are based on the coordinate system of the first image, and in FIG. 12(b), the point coordinates of the modified state are based on the coordinate system of the second image. When the coordinates are different between the first image and the second image, conversion of either coordinate is required when performing the superimposed display. Therefore, for example, in order to perform alignment between the first image and the second image, coordinate conversion may be performed using the following formula (1).

[0053] [Number]

[0054] (x 1 , y 1 ) and (x 2 , y 2 ) in Equation (1) represent the coordinates of the first image and the coordinates of the second image, respectively. Also, the constants a to f are constants calculated based on the coordinates of the vertices of the image.

[0055] On the other hand, when switching between the first image and the second image, if coordinate conversion is performed using Equation (1) by the display control unit 104, the load of the calculation process increases, and thus smooth image switching may not be possible. Therefore, when acquiring the change over time, the change over time acquisition unit 103 may calculate in advance using Equation (1) and hold the coordinates of the changed points in both the coordinate system of the first image and the coordinate system of the second image.

[0056] FIG. 13 is a diagram showing an example of the common changed state and differential changed state data of the change over time acquired by the change over time acquisition unit 103 from the changed state data shown in FIG. 12. FIG. 13(a) represents the differential changed state data included in the first image but not included in the second image. FIG. 13(b) represents the common changed state data included in the first image and the second image and detected from the first image. Also, FIG. 13(c) represents the common changed state data included in the first image and the second image and detected from the second image, and FIG. 13(d) represents the differential changed state data included in the second image but not included in the first image.

[0057] Here, the differential deformation data of the first image shown in FIG. 13(a) and the common deformation data of the first image shown in FIG. 13(b) are classified by temporal changes from the first deformation detected in the first image. Therefore, although the point coordinates of the deformation are held in the coordinate system of the first image, the point coordinates in the coordinate system of the second image are also held by pre-calculating with Equation (1). Similarly, the common deformation data of the second image shown in FIG. 13(c) and the differential deformation data of the second image shown in FIG. 13(d) are classified by temporal changes from the second deformation detected in the second image. Therefore, although the point coordinates of the deformation are held in the coordinate system of the second image, the point coordinates in the coordinate system of the first image are also held by pre-calculating with Equation (1).

[0058] As described above, according to the present embodiment, the temporal change acquisition unit 103 acquires the point coordinates in both the coordinate system of the first image and the coordinate system of the second image as data of temporal changes. Thereby, even when there is a difference in the coordinate system between the first image and the second image, when the display control unit 104 switches between the first image and the second image and superimposes and displays the deformation information, there is no need to perform coordinate conversion, so that the superimposed display can be switched at high speed.

[0059] (Sixth Embodiment) In the first embodiment described above, the display control unit 104 switches between and displays the first image and the second image, and at that time, changes the display mode of the deformation over time and superimposes and displays it. On the other hand, in order to display more quickly when switching the images, the display control unit 104 may pre-determine the deformation and the display mode to be superimposed on the other image that is not being displayed when the first image or the second image is being displayed, and pre-generate the superimposed image. For example, by pre-generating the superimposed image shown in FIG. 7(b) while the superimposed image shown in FIG. 7(a) is being displayed, the superimposed image can be displayed more quickly without delay when switching to and displaying the second image.

[0060] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and having one or more processors in a computer of the system or apparatus read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0061] The disclosure of this embodiment includes the following configurations, methods, and programs.

[0062] (Configuration 1) An acquisition means for acquiring a change over time in the state change of an object between a first image obtained by photographing the object and a second image obtained by photographing the object at a time different from the first image; A display control means for displaying the state change on a display device in a display mode corresponding to the change over time in the state change acquired by the acquisition means, by displaying the first image or the second image; and the display control means is characterized in that the display mode of the state change is made different between the first image and the second image. An image processing apparatus.

[0063] (Configuration 2) further having a state change acquisition means for acquiring the state change of the object from each of the first image and the second image; The acquisition means is characterized in that it acquires a change over time in the state change respectively acquired from the first image and the second image by the state change acquisition means. The image processing apparatus according to Configuration 1. (Configuration 3) The image processing apparatus according to Configuration 1 or 2, further comprising an instruction means for instructing switching between the display of the first image and the second image by the display control means. (Configuration 4) The acquisition means acquires, as the change over time, a differential change that is a change not included in either the first image or the second image, and a common change that is a change included in both the first image and the second image, and the image processing apparatus according to any one of Configurations 1 to 3 is characterized in this.

[0064] (Configuration 5) The display control means makes the display mode different depending on the differential change included in the image to be displayed and the differential change not included in the image to be displayed, and the image processing apparatus according to Configuration 4 is characterized in this. (Configuration 6) The display control means displays the common change included in the image to be displayed and does not display the common change included in the other image that is not being displayed, and the image processing apparatus according to Configuration 4 or 5 is characterized in this. (Configuration 7) When the difference in the common change between the first image and the second image is equal to or greater than a predetermined value, the display control means displays both the common change included in the image to be displayed and the common change included in the other image that is not being displayed, and the image processing apparatus according to Configuration 4 or 5 is characterized in this.

[0065] (Configuration 8) The image processing apparatus further has a designation means for designating a position in the image being displayed by the display control means. In a partial region corresponding to the position designated by the designation means, the display control means displays the change in the display mode of the other image that is not being displayed, and the image processing apparatus according to any one of Configurations 1 to 7 is characterized in this. (Configuration 9) The acquisition means acquires the change over time of the change in the object from an external device, and the image processing apparatus according to any one of Configurations 1 to 8 is characterized in this. (Configuration 10) The acquisition means further acquires the coordinates of the differential change and the common change in both the coordinate system of the first image and the coordinate system of the second image, and the image processing apparatus according to any one of Configurations 4 to 7 is characterized in this. (Configuration 11) The image processing apparatus according to any one of Configurations 1 to 10, wherein the display control means determines a display mode of a change in the other image that is not being displayed when displaying either the first image or the second image.

[0066] (Method) An acquisition step of acquiring a change over time in a change in the object between a first image obtained by photographing the object and a second image obtained by photographing the object at a time different from the first image; A display control step of displaying the first image or the second image on a display device so as to display the change in a display mode according to the change over time in the change acquired in the acquisition step; having A control method for an image processing apparatus, wherein in the display control step, the display mode of the change is made different between the first image and the second image. (Program) A program for causing a computer to function as each means of the image processing apparatus according to any one of Configurations 1 to 11.

Description of Reference Numerals

[0067] 103 Change-over-time acquisition unit, 104 Display control unit

Claims

1. An acquisition means for acquiring the change over time of the change in the object between a first image of the object taken and a second image of the object taken at a time different from the first image; A display control means for displaying the first image or the second image on a display device so as to display the change in a display mode corresponding to the change over time of the change acquired by the acquisition means; comprising: The display control means is characterized in that the display mode of the change is made different between the first image and the second image. An image processing apparatus.

2. Further comprising a change acquisition means for acquiring the change in the object from each of the first image and the second image; The acquisition means is characterized in that the acquisition means acquires the change over time of the change respectively acquired from the first image and the second image by the change acquisition means. The image processing apparatus according to claim 1.

3. The image processing apparatus according to claim 1, further comprising an instruction means for instructing switching of display between the first image and the second image by the display control means.

4. The acquisition means is characterized in that, as the change over time, it acquires a differential change which is a change not included in either one of the first image and the second image, and a common change which is a change included in both the first image and the second image. The image processing apparatus according to claim 1.

5. The display control means is characterized in that the display mode is made different between the differential change included in the image to be displayed and the differential change not included in the image to be displayed. The image processing apparatus according to claim 4.

6. The display control means is characterized in that it displays the common change included in the image to be displayed and does not display the common change included in the other image that is not being displayed. The image processing apparatus according to claim 4.

7. The display control means is characterized in that when the difference in the common change between the first image and the second image is equal to or greater than a predetermined value, it displays both the common change included in the image to be displayed and the common change included in the other image that is not being displayed. The image processing apparatus according to claim 4.

8. Further comprising a designation means for designating a position in the image being displayed by the display control means; The image processing apparatus according to claim 1, wherein the display control means displays a change in the display mode of the other image that is not being displayed in a partial region corresponding to the position specified by the specifying means.

9. The image processing apparatus according to claim 1, wherein the acquisition means acquires the change over time of the change in the object from an external device.

10. The image processing apparatus according to claim 4, wherein the acquisition means further acquires the coordinates of the differential change and the common change in both the coordinate system of the first image and the coordinate system of the second image.

11. The image processing apparatus according to claim 1, wherein the display control means determines the display mode of the change in the other image that is not being displayed when displaying either the first image or the second image.

12. An acquisition step of acquiring the change over time of the change in the object between a first image obtained by photographing the object and a second image obtained by photographing the object at a different time from the first image; A display control step of displaying the first image or the second image on a display device so as to display the change in a display mode corresponding to the change over time of the change acquired in the acquisition step; comprising A control method for an image processing apparatus, wherein in the display control step, the display mode of the change is made different between the first image and the second image.

13. A program for causing a computer to function as each means of the image processing apparatus according to claim 1.

Images