Information processing device, information processing method, and information processing program

Abstract

An information processing device (100) comprises: an acquisition unit (120) that acquires a first image and a second image; and a generation processing unit (140) that generates a first difference image indicating a difference obtained by simply overlaying the first image and the second image, aligns an object included in a target image, which is an image selected from among the first image and the second image, with an object included in a non-target image, which is the image not selected as the target image, thereby generating a first alignment image based on the target image, generates a second difference image indicating a difference obtained by simply overlaying the first alignment image and the non-target image, and generates a reference image indicating portions common to the first difference image and the second difference image. The first image and the second image are images in which the shape and position of an object differ due to differing imaging directions, or the like, and in which different distortions occur due to compositing, wide-angle imaging, and the like.

Description

Information processing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program.

[0002] Work is performed on a panel. For example, the panel is a control panel. After the work is completed, the worker may want to know what changes have been made. A technology for detecting changes has been proposed (see Patent Document 1). The change detection device in Patent Document 1 detects changes in the appearance of a control panel.

[0003] Japanese Patent Application Laid-Open No. 2021-103562

[0004] The above-described technique for detecting changes uses two images. The two images may have different shapes and positions of objects due to different shooting directions, and may also have different distortions due to compositing or wide-angle shooting. In such cases, the above-described technique has low accuracy in detecting changes.

[0005] An object of the present disclosure is to improve the accuracy of detecting changes.

[0006] According to one aspect of the present disclosure, there is provided an information processing device. The information processing device includes an acquisition unit that acquires a first image and a second image, and a generation processing unit that generates a first difference image showing a difference obtained by simply superimposing the first image and the second image, aligns an object included in a target image, which is an image selected from the first image and the second image, with an object included in a non-target image, which is an image not selected as the target image, to generate a first aligned image based on the target image, generates a second difference image showing a difference obtained by simply superimposing the first aligned image and the non-target image, and generates a reference image showing a common portion between the first difference image and the second difference image. The first image and the second image are images in which the shapes and positions of objects are different due to, for example, different shooting directions, and in which different distortions occur due to, for example, compositing or wide-angle shooting.

[0007] According to the present disclosure, it is possible to improve the accuracy of detecting changes.

[0008] FIG. 1 is a diagram showing hardware included in an information processing device of embodiment 1. FIG. 2 is a block diagram showing functions of the information processing device of embodiment 1. FIG. 3 is a diagram showing a specific example of processing performed by an information processing device of embodiment 1. FIG. 4 is a diagram showing a specific example of processing performed by an information processing device of embodiment 2. FIG. 5 is a diagram showing a specific example of a modification of embodiment 2. FIG. 6 is a diagram showing a specific example of processing performed by an information processing device of embodiment 3. FIG. 7 is a diagram showing a specific example of processing performed by an information processing device of embodiment 4. FIG. 8 is a diagram showing a specific example of processing performed by an information processing device of embodiment 5. FIG. 9 is a diagram showing a specific example of processing performed by an information processing device of embodiment 6. FIG. 10 is a block diagram showing functions of an information processing device of embodiment 7. FIG. 11 is a diagram showing a specific example of processing performed by an information processing device of embodiment 7.

[0009] Hereinafter, embodiments will be described with reference to the drawings. The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1.

[0011] FIG. 1 is a diagram showing hardware included in an information processing device according to a first embodiment. The information processing device 100 is a device that executes an information processing method. For example, the information processing device 100 is a personal computer (PC). The information processing device 100 may also be a smartphone, a tablet device, or a server. The information processing device 100 includes a processor 101, a volatile storage device 102, and a non-volatile storage device 103.

[0012] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.

[0013] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a random access memory (RAM). The nonvolatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the nonvolatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD).

[0014] Next, a description will be given of functions of the information processing device 100. Fig. 2 is a block diagram showing functions of the information processing device of embodiment 1. The information processing device 100 has a storage unit 110, an acquisition unit 120, an image processing unit 130, a generation processing unit 140, and an output unit 150.

[0015] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103. Some or all of the acquisition unit 120, image processing unit 130, generation processing unit 140, and output unit 150 may be realized by processing circuits. Furthermore, some or all of the acquisition unit 120, image processing unit 130, generation processing unit 140, and output unit 150 may be realized as program modules executed by the processor 101. For example, the program executed by the processor 101 is also referred to as an information processing program. For example, the information processing program is recorded on a recording medium.

[0016] The storage unit 110 stores various information. The functions of the acquisition unit 120, the image processing unit 130, the generation processing unit 140, and the output unit 150 will be described in detail later.

[0017] Next, a specific example will be used to explain the processing executed by the information processing device 100. Fig. 3 is a diagram showing a specific example of the processing executed by the information processing device of embodiment 1. In the specific example, a panel is photographed, two images are generated by the photographing, and changes are detected using the two images. The panel may be a distribution panel, a power panel, a control panel, or the like.

[0018] 3 shows image 10 and image 20. Image 20 is an image generated after image 10. For example, image 10 is an image showing the state of the board before work is performed on it. Image 20 is an image showing the state after work is performed on it. Image 10 is also referred to as the first image. Image 20 is also referred to as the second image.

[0019] Image 10 includes object 11 and object 12. Image 20 includes object 21. Object 11 and object 21 are the same object. In this way, image 20 includes the same object as the object included in image 10.

[0020] For example, image 10 is generated by photographing the board from the left side of the board. Image 20 is generated by photographing the board from the right side of the board. In this way, the position of object 11 in image 10 and the position of object 21 in image 20 differ due to the difference in photographing direction.

[0021] Here, the board is large. Therefore, it is difficult to photograph the entire board in one shot. Therefore, multiple shots are taken. Then, the multiple images obtained by photographing are combined to obtain an image showing the entire board. Images 10 and 20 can be considered as a composite image. When images 10 and 20 are composite images, the composite accuracy of image 20 may differ from that of image 10 due to differences in the photographing method, etc. Therefore, the shapes of object 11 and object 21 are different.

[0022] As mentioned above, the board is large. By using an ultra-wide-angle camera, it is possible to capture the entire board in one shot. However, distortion occurs in the generated image. Images 10 and 20 can be considered as images with distortion. If images 10 and 20 are distorted due to differences in the shooting methods, the degree of distortion in image 20 may differ from the degree of distortion in image 10. Therefore, the shapes of object 11 and object 21 are different.

[0023] The acquisition unit 120 acquires the image 10 and the image 20. For example, the acquisition unit 120 acquires the image 10 and the image 20 from the storage unit 110. Furthermore, for example, the acquisition unit 120 acquires the image 10 and the image 20 from an external device. Note that the external device is a device that exists outside the information processing device 100. For example, the external device is a cloud server, an external memory, etc. The external device is not shown in the drawing.

[0024] If a shadow exists in the images 10 and 20, the image processing unit 130 may remove the shadow. For example, the image processing unit 130 may remove the shadow using a method such as Sauvola. The image processing unit 130 may perform white balance correction on the images 10 and 20. The image processing unit 130 may also perform grayscale correction on the images 10 and 20.

[0025] The generation processing unit 140 simply generates a difference image 40 that indicates the difference obtained by superimposing the image 10 and the image 20. The generation processing will be described in detail. State 30 is a state in which the image 10 and the image 20 are simply superimposed. The generation processing unit 140 calculates the difference in pixel values ​​between corresponding pixels in the superimposed state of the image 10 and the image 20. Note that the pixel value is a value expressed by RGB, luminance, brightness, saturation, hue, or the like. The value indicated by the difference may also be expressed as an absolute value. If the difference is equal to or greater than a threshold, the generation processing unit 140 sets the pixel in the difference image 40 corresponding to the pixel for which the difference was calculated to white. If the difference is smaller than the threshold, the generation processing unit 140 sets the pixel in the difference image 40 corresponding to the pixel for which the difference was calculated to black. The generation processing unit 140 performs the same processing on all pixels in the image 10 and the image 20. In this way, the difference image 40 is generated. Note that the difference image 40 is also referred to as a first difference image.

[0026] The difference image 40 is represented in two colors: black and white, but the difference image 40 may also be represented in two or more colors.

[0027] The generation processing unit 140 aligns an object included in a target image, which is an image selected from image 10 and image 20, with an object included in a non-target image, which is an image not selected as the target image, to generate an aligned image based on the target image. In the following description, image 10 is the target image. Image 20 is the non-target image. An example of the generation process will be described in detail. The generation processing unit 140 divides image 10 to set multiple divided regions. FIG. 3 shows that 16 divided regions have been set. The generation processing unit 140 aligns each divided region. An example of aligning a divided region containing object 11 will be described. FIG. 3 simply shows a state in which object 11 in image 10 and object 21 in image 20 are superimposed. The generation processing unit 140 searches for a state in which the difference between the pixel value of the divided region (i.e., the pixel value of object 11) and the pixel value within frame 22 is smallest. Note that pixel values ​​are values ​​expressed in RGB, brightness, lightness, saturation, hue, etc. In this way, the generation processing unit 140 moves the divided area where the object 11 is located to search for the state where the difference is smallest. After the search is completed, the generation processing unit 140 detects the distance and direction between the position of the divided area where the object 11 is located in the image 10 and the position of the divided area where the difference is smallest. The generation processing unit 140 moves the position of the divided area where the object 11 is located in the image 10 based on the distance and direction, thereby generating an aligned image 50 including the divided area. The aligned image 50 indicates that the divided area where the object 11 is located in the image 10 has moved to the right. The generation processing unit 140 performs similar processing on the remaining 15 divided areas. By performing alignment on all divided areas, a new image, the aligned image 50, is generated. In other words, the aligned image 50 based on the image 10 is generated. The aligned image 50 is also referred to as a first aligned image.

[0028] In the above, a case has been described in which the area within the frame 22 surrounding the object 21 is the search target area. The search target area is not limited to the area within the frame 22. For example, the search target area may be the entire image 20. When performing alignment, the generation processing unit 140 may use a phase-only correlation method, a "template matching in pyramids", or the like.

[0029] In the above description, a case has been described in which a plurality of divided regions are set in the image 10, and the generation processing unit 140 aligns the plurality of divided regions with the image 20. The generation processing unit 140 may set a plurality of divided regions in the image 20, and align the plurality of divided regions with the image 10. Then, the generation processing unit 140 may generate an aligned image in which the image 20 has been changed.

[0030] The generation processing unit 140 simply generates a difference image 70 that indicates the difference obtained by superimposing the aligned image 50 and the image 20. An example of the generation process will be described in detail. State 60 is a state in which the aligned image 50 and the image 20 are simply superimposed. The generation processing unit 140 calculates the difference in pixel values ​​between corresponding pixels in the superimposed aligned image 50 and the image 20. Note that the pixel value is a value expressed by RGB, luminance, brightness, saturation, hue, or the like. The value indicated by the difference may also be expressed as an absolute value. If the difference is greater than or equal to a threshold, the generation processing unit 140 sets the pixel in the difference image 70 corresponding to the pixel for which the difference was calculated to white. If the difference is smaller than the threshold, the generation processing unit 140 sets the pixel in the difference image 70 corresponding to the pixel for which the difference was calculated to black. The generation processing unit 140 performs the same process on all pixels in the aligned image 50 and the image 20. In this way, the difference image 70 is generated. Note that the difference image 70 is also referred to as a second difference image.

[0031] When an aligned image in which image 20 has been modified is generated, generation processing unit 140 generates difference image 70 that shows the difference obtained by superimposing the aligned image and image 10 .

[0032] The generation processing unit 140 generates a reference image 80 that shows a common portion 81 between the difference image 40 and the difference image 70 .

[0033] The generation processing unit 140 may remove noise if the reference image 80 contains noise. For example, the generation processing unit 140 removes noise using morphological transformation, a smoothing filter, or the like.

[0034] For example, the image processing unit 130 compares the reference image 80 with the image 10, and detects a portion of the image 10 that corresponds to the common portion 81 (i.e., the object 12) as a change point. The image processing unit 130 may mark the change point. For example, the image processing unit 130 marks the change point with a red frame.

[0035] Furthermore, for example, the image processing unit 130 compares the reference image 80 with the image 20, and detects, in the image 20, a portion that corresponds to the common portion 81 as a change point. The image processing unit 130 may mark the change point. For example, the image processing unit 130 marks the change point with a red frame. In this way, the image processing unit 130 can detect the change point.

[0036] The output unit 150 may output the marked image. For example, the output unit 150 outputs the image to a display of the information processing device 100.

[0037] According to the first embodiment, the information processing device 100 generates information for detecting changes (i.e., a reference image 80 including a common portion 81) using two images 10 and 20, each of which is an image in which the shapes and positions of objects are different due to factors such as different shooting directions and in which different distortions have occurred due to compositing or wide-angle shooting. The information processing device 100 can appropriately detect changes by using the reference image 80. Therefore, by generating the reference image 80, the information processing device 100 can improve the accuracy of detecting changes.

[0038] Second Embodiment Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be mainly described. Furthermore, in the second embodiment, descriptions of the commonalities between the first embodiment and the second embodiment will be omitted.

[0039] Fig. 4 is a diagram showing a specific example of processing executed by the information processing device of embodiment 2. Fig. 4 shows images 200 and 210. Image 210 is an image generated after image 200. For example, image 200 is an image showing the state before work is performed on the board. Image 210 is an image showing the state after work is performed. Image 200 is also referred to as the first image. Image 210 is also referred to as the second image.

[0040] Image 200 includes object 201 and object 202. Image 210 includes object 211. Object 201 and object 211 are the same object. In this way, image 210 includes the same object as the object included in image 200. Images 200 and 210 are images in which the shapes and positions of the objects are different due to differences in the shooting directions, and in which different distortions have occurred due to compositing, wide-angle shooting, etc.

[0041] The generation processing unit 140 simply generates a difference image 230 that indicates the difference obtained by superimposing the image 200 and the image 210. The generation processing is the same as the processing for generating the difference image 40 in the first embodiment. Therefore, the generation processing will be briefly described. State 220 is simply a state in which the image 200 and the image 210 are superimposed. The generation processing unit 140 calculates the difference in pixel values ​​between corresponding pixels in the state in which the image 200 and the image 210 are superimposed. The generation processing unit 140 performs the same processing on all pixels in the image 200 and the image 210. In this way, the difference image 230 is generated. The difference image 230 is also referred to as a first difference image.

[0042] The generation processing unit 140 generates a registered image 240 by aligning an object included in the image 200 with an object included in the image 210. When performing the alignment, the generation processing unit 140 also performs the alignment taking into consideration areas of the non-target image that correspond to difference areas 231, 232, and 233, which are difference areas indicated by the difference image 230.

[0043] The generation process will be described in detail. The target image is image 200. The non-target image is image 210. The generation processing unit 140 divides the image 200 to set multiple divided regions. FIG. 4 shows that 16 divided regions have been set. The generation processing unit 140 aligns each divided region. The alignment of the divided region in which the object 201 exists will be described. FIG. 4 shows regions 212, 213, and 214 corresponding to difference regions 231, 232, and 233. The regions corresponding to the difference regions may be called weight regions. The generation processing unit 140 overlays the divided regions on the image 210 and searches for the state in which the value calculated by equation (1) is smallest.

[0044] value=difference from pixel value of weighted region / area of ​​weighted region+difference from pixel value of divided region / area of ​​divided region (1)

[0045] Specifically, the generation processing unit 140 overlays the divided region where the object 201 exists on the image 210 and searches for the state where the value calculated by equation (2) is smallest. The generation processing unit 140 also performs a search within a frame 215. The region within the frame 215 is the search target region. Furthermore, the pixel value is a value expressed by RGB, luminance, brightness, saturation, hue, etc.

[0046] Value=(difference from pixel value of region 212+difference from pixel value of region 213) / (area of ​​region 212+area of ​​region 213)+difference from pixel value of divided region in which object 201 exists / area of ​​divided region in which object 201 exists (2)

[0047] In this way, the generation processing unit 140 moves the divided area in which the object 201 exists and performs alignment so that the value becomes the smallest. The generation processing unit 140 performs the same process on the remaining 15 divided areas.

[0048] Furthermore, if the area corresponding to the differential area (that is, the weight area) is not included in the search target area, the generation processing unit 140 searches for the state in which the value calculated by equation (3) is smallest.

[0049] value=difference from pixel value of divided region / area of ​​divided region (3)

[0050] By performing the alignment for all the divided regions, a new image, an aligned image 240, is generated. The aligned image 240 is also referred to as a first aligned image.

[0051] As in the first embodiment, the search target area is not limited to the area within the frame 215. Furthermore, the generation processing unit 140 may set a plurality of divided areas in the image 210 and align the plurality of divided areas with the image 200. Then, the generation processing unit 140 may generate an aligned image in which the image 210 has been changed.

[0052] The generation processing unit 140 simply generates a difference image 260 that indicates the difference obtained by superimposing the aligned image 240 and the image 210. The generation processing is the same as the processing for generating the difference image 70 in embodiment 1. Therefore, the generation processing will be briefly described. State 250 is simply a state in which the aligned image 240 and the image 210 are superimposed. The generation processing unit 140 calculates the difference in pixel values ​​between corresponding pixels in the state in which the aligned image 240 and the image 210 are superimposed. The generation processing unit 140 performs the same processing on all pixels in the aligned image 240 and the image 210. In this way, the difference image 260 is generated. The difference image 260 is also referred to as a second difference image.

[0053] When an aligned image in which the image 210 has been modified is generated, the generation processing unit 140 generates a difference image 260 that shows the difference obtained by superimposing the aligned image and the image 200 .

[0054] The generation processing unit 140 generates a reference image 270 that shows a common portion 271 between the difference image 230 and the difference image 260 .

[0055] According to the second embodiment, the information processing device 100 performs alignment taking into consideration the area corresponding to the difference area indicated by the difference image 230. This enables the information processing device 100 to further improve the accuracy of detecting changes.

[0056] Variation of the Second Embodiment As described in the second embodiment, the generation processing unit 140 overlays the divided region in which the object 201 exists on the image 210 and searches for the state in which the value calculated by the formula (1) is smallest. The formula (1) uses a weight region. In the variation of the second embodiment, a case will be described in which a peripheral region around the edge of the weight region is used.

[0057] FIG. 5 is a diagram showing a specific example of a modification of the second embodiment. FIG. 5 shows an example of a case where alignment processing is performed. In the modification of the second embodiment, a peripheral region of an edge of a weight region is used. For example, a peripheral region of an edge 212a of a weight region 212 is used. The peripheral region is a region outside the region 212. Also, for example, a peripheral region of an edge 213a of a weight region 213 is used. The peripheral region is a region outside the region 213. For example, the peripheral region is a region up to one pixel away from the edge.

[0058] The generation processing unit 140 overlays the divided regions on the image 210 and searches for the state in which the value calculated by equation (4) is smallest.

[0059] value=difference between pixel values ​​of surrounding areas of weighting area / area of ​​surrounding areas of weighting area+difference between pixel values ​​of divided areas / area of ​​divided areas (4)

[0060] Specifically, the generation processing unit 140 overlays the divided region in which the object 201 exists on the image 210, and searches for the state in which the value calculated by equation (5) is smallest.

[0061] Value = (difference between pixel values ​​of the surrounding area of ​​the area 212 + difference between pixel values ​​of the surrounding area of ​​the area 213) / (area of ​​the surrounding area of ​​the area 212 + area of ​​the surrounding area of ​​the area 213) + difference between pixel values ​​of the divided area where the object 201 exists / area of ​​the divided area where the object 201 exists ... (5)

[0062] In this way, when performing registration, the generation processing unit 140 performs registration taking into consideration the peripheral area of ​​the area of ​​the non-target image (i.e., the weight area) that corresponds to the difference area indicated by the difference image 230.

[0063] Furthermore, when the generation processing unit 140 sets a plurality of divided regions in the image 210 and aligns the plurality of divided regions with the image 200, the alignment may be performed using a peripheral region of the weight region.

[0064] According to the modification of the second embodiment, the information processing device 100 performs alignment taking into consideration the surrounding area of ​​the area corresponding to the difference area indicated by the difference image 230. This enables the information processing device 100 to further improve the accuracy of detecting changes.

[0065] Embodiment 3 Next, embodiment 3 will be described. In embodiment 3, differences from embodiments 1 and 2 will be mainly described. Furthermore, in embodiment 3, descriptions of matters common to embodiments 1 and 2 will be omitted.

[0066] 6 is a diagram showing a specific example of processing executed by the information processing device of embodiment 3. In embodiment 3, a difference image 40 is not generated as in embodiment 1. In embodiment 3, a difference image 70 is generated as in embodiment 1. Note that the difference image 70 of embodiment 3 is also referred to as a first difference image. The difference image 70 includes difference regions 71, 72, and 73.

[0067] The generation processing unit 140 aligns the objects contained in the target image with the objects contained in the non-target image, and performs the alignment taking into account the areas of the non-target image that correspond to the difference areas shown in the difference image 70, thereby generating an aligned image 90.

[0068] The generation process will be described in detail. The target image is image 10. The non-target image is image 20. FIG. 6 also shows regions 23, 24, and 25 corresponding to differential regions 71, 72, and 73. When performing alignment, the generation processing unit 140 performs alignment taking into account regions 23, 24, and 25 corresponding to differential regions 71, 72, and 73, as in the second embodiment. In the search for alignment, the generation processing unit 140 searches for the state in which the value calculated by equation (1) is smallest.

[0069] The alignment of the divided region where the object 11 is present will now be described. The generation processing unit 140 overlays the divided region where the object 11 is present on the image 20, and searches for the state where the value calculated by equation (6) is the smallest. The region within the frame 22 is the region to be searched.

[0070] value=(difference from pixel value of region 23+difference from pixel value of region 24) / (area of ​​region 23+area of ​​region 24)+difference from pixel value of divided region in which object 11 exists / area of ​​divided region in which object 11 exists (6)

[0071] In this way, the generation processing unit 140 moves the divided area in which the object 11 exists and performs alignment so that the value becomes the smallest.

[0072] Furthermore, if the search target region does not include a region corresponding to the differential region, the generation processing unit 140 searches for the state in which the value calculated by equation (3) is smallest.

[0073] The generation processing unit 140 performs the same process on the remaining 15 divided regions. By performing alignment on all divided regions, a new image, that is, an aligned image 90, is generated. In other words, the aligned image 90 is generated based on the image 10. The aligned image 90 is also referred to as a second aligned image.

[0074] As in the first embodiment, the search target area is not limited to the area within the frame 22. Furthermore, the generation processing unit 140 may set a plurality of divided areas in the image 20 and align the plurality of divided areas with the image 10. The generation processing unit 140 may then generate an aligned image in which the image 20 has been modified. Furthermore, the generation processing unit 140 may perform alignment using a modified example of the second embodiment.

[0075] The generation processing unit 140 simply generates a difference image 92 that shows the difference obtained by superimposing the aligned image 90 and the image 20. The generation processing is the same as the processing for generating the difference image 70 in the first embodiment. Therefore, the generation processing will be briefly explained. State 91 is simply a state in which the aligned image 90 and the image 20 are superimposed. The generation processing unit 140 calculates the difference in pixel values ​​between corresponding pixels in the state in which the aligned image 90 and the image 20 are superimposed. The generation processing unit 140 performs the same processing on all pixels in the aligned image 90 and the image 20. As a result, the difference image 92 is generated.

[0076] When an aligned image in which image 20 has been modified is generated, generation processing unit 140 generates difference image 92 that shows the difference obtained by superimposing the aligned image and image 10. Note that difference image 92 is also referred to as a second difference image.

[0077] The generation processing unit 140 generates a reference image 93 that shows a common portion 93 a between the difference image 70 and the difference image 92 .

[0078] According to the third embodiment, the information processing device 100 generates information for detecting changes (i.e., a reference image 93 including a common portion 93a) using two images 10 and 20, each of which is an image in which the shapes and positions of objects are different due to factors such as different shooting directions and in which different distortions have occurred due to compositing or wide-angle shooting. The information processing device 100 can appropriately detect changes by using the reference image 93. Therefore, by generating the reference image 93, the information processing device 100 can improve the accuracy of detecting changes.

[0079] Embodiment 4 Next, embodiment 4 will be described. In embodiment 4, differences from embodiments 1 and 2 will be mainly described. Furthermore, in embodiment 4, descriptions of matters common to embodiments 1 and 2 will be omitted.

[0080] Fig. 7 is a diagram showing a specific example of processing executed by the information processing device of embodiment 4. Fig. 7 shows images 300 and 310. Image 310 is an image generated after image 300. For example, image 300 is an image showing the state before work is performed on the board. Image 310 is an image showing the state after work is performed. Image 300 is also referred to as the first image. Image 310 is also referred to as the second image.

[0081] Image 300 includes object 301 and object 302. Image 310 includes object 311. Object 301 and object 311 are the same object. In this way, image 310 includes the same object as the object included in image 300. Images 300 and 310 are images in which the shapes and positions of the objects are different due to factors such as different shooting directions, and in which different distortions have occurred due to compositing, wide-angle shooting, etc.

[0082] In the following description, the target image is image 300, and the non-target image is image 310.

[0083] The generation processing unit 140 generates a difference image 320 that shows the difference obtained by simply superimposing the image 300 and the image 310. The generation process is the same as the process for generating the difference image 40 in the first embodiment. Therefore, a description of the generation process will be omitted. The difference image 320 is also referred to as a first difference image.

[0084] The generation processing unit 140 generates a registered image by aligning an object included in the image 300 with an object included in the image 310. When performing this alignment, the generation processing unit 140 also performs the alignment taking into account the area of ​​the image 310 that corresponds to the difference area indicated by the difference image 320. The generated registered image is also referred to as a second registered image. The generation processing unit 140 simply generates a difference image 330 that indicates the difference obtained by superimposing the registered image and the image 310. The difference image 330 is also referred to as a third difference image. The process of generating this registered image and the process of generating the difference image 330 are the same as the process of generating the registered image 240 and the process of generating the difference image 260 in the second embodiment. Therefore, a detailed description of the generation process will be omitted.

[0085] The generation processing unit 140 generates an aligned image by aligning an object included in the image 300 with an object included in the image 310. The generation processing unit 140 simply generates a difference image 340 that shows the difference obtained by superimposing the aligned image and the image 310. The generation process of the aligned image and the generation process of the difference image 340 are the same as the generation process of the aligned image 50 and the generation process of the difference image 70 in the first embodiment. Therefore, a detailed description of the generation process will be omitted. The difference image 340 is also referred to as a second difference image.

[0086] The generation processing unit 140 generates a reference image 350 that shows a common portion 351 between the difference image 320 , the difference image 330 , and the difference image 340 .

[0087] According to the fourth embodiment, the information processing device 100 can further improve the accuracy of detecting changes, similarly to the second embodiment.

[0088] Embodiment 5 Next, embodiment 5 will be described. In embodiment 5, differences from embodiments 1 and 3 will be mainly described. Furthermore, in embodiment 5, descriptions of matters common to embodiments 1 and 3 will be omitted.

[0089] 8 is a diagram showing a specific example of processing executed by the information processing device of embodiment 5. FIG. 8 shows images 400 and 410. Image 410 is an image generated after image 400. For example, image 400 is an image showing the state of a board before work is performed on it. Image 410 is an image showing the state after work is performed on it.

[0090] Image 400 includes object 401 and object 402. Image 410 includes object 411. Object 401 and object 411 are the same object. In this way, image 410 includes the same object as the object included in image 400. Images 400 and 410 are images in which the shapes and positions of objects are different due to different shooting directions, and in which different distortions have occurred due to compositing, wide-angle shooting, etc. Image 400 is also referred to as a first image. Image 410 is also referred to as a second image.

[0091] In the following description, the target image is the image 400, and the non-target image is the image 410.

[0092] The generation processing unit 140 generates a difference image 420 that shows the difference obtained by simply superimposing the image 400 and the image 410. The generation process is the same as the process for generating the difference image 40 in the first embodiment. Therefore, a description of the generation process will be omitted. The difference image 420 is also referred to as a third difference image.

[0093] The generation processing unit 140 generates an aligned image by aligning an object included in the image 400 with an object included in the image 410. The generation processing unit 140 simply generates a difference image 430 that shows the difference obtained by superimposing the aligned image and the image 410. The generation process of the aligned image and the generation process of the difference image 430 are the same as the generation process of the aligned image 50 and the generation process of the difference image 70 in the first embodiment. Therefore, a detailed description of the generation process will be omitted. The difference image 430 is also referred to as a first difference image.

[0094] The generation processing unit 140 generates a registered image by aligning an object included in the image 400 with an object included in the image 410. Furthermore, when performing this alignment, the generation processing unit 140 performs the alignment taking into account the area of ​​the image 410 that corresponds to the difference area indicated by the difference image 430. The generation processing unit 140 generates a difference image 440 that indicates the difference obtained by simply superimposing the registered image and an image that has not been aligned. The process of generating this registered image and the process of generating the difference image 440 are the same as the process of generating the registered image 90 and the process of generating the difference image 92 in embodiment 3. Therefore, a detailed description of the generation process will be omitted. The difference image 440 is also referred to as a second difference image.

[0095] The generation processing unit 140 generates a reference image 450 that shows a common portion 451 between the difference image 420 , the difference image 430 , and the difference image 440 .

[0096] According to the fifth embodiment, the information processing device 100 can further improve the accuracy of detecting changes.

[0097] Sixth Embodiment Next, a sixth embodiment will be described. In the sixth embodiment, differences from the first to third embodiments will be mainly described. In the sixth embodiment, descriptions of the commonalities between the first to third embodiments will be omitted.

[0098] 9 is a diagram showing a specific example of processing executed by the information processing device of embodiment 6. FIG. 9 shows images 500 and 510. Image 510 is an image generated after image 500. For example, image 500 is an image showing the state of a board before work is performed on it. Image 510 is an image showing the state after work is performed on it.

[0099] Image 500 includes object 501 and object 502. Image 510 includes object 511. Object 501 and object 511 are the same object. In this way, image 510 includes the same object as the object included in image 500. Image 500 is also referred to as a first image. Image 510 is also referred to as a second image. Images 500 and 510 are images in which the shapes and positions of objects are different due to different shooting directions, and in which different distortions have occurred due to compositing, wide-angle shooting, etc.

[0100] In the following description, the target image is the image 500, and the non-target image is the image 510.

[0101] The generation processing unit 140 generates a difference image 520 that shows the difference obtained by simply superimposing the image 500 and the image 510. The generation process is the same as the process for generating the difference image 40 in the first embodiment. Therefore, a description of the generation process will be omitted. The difference image 520 is also referred to as a first difference image.

[0102] The generation processing unit 140 generates a registered image by aligning an object included in the image 500 with an object included in the image 510. When performing this alignment, the generation processing unit 140 also performs the alignment taking into account the area of ​​the image 510 that corresponds to the difference area indicated by the difference image 520. The generation processing unit 140 simply generates a difference image 530 that indicates the difference obtained by superimposing the registered image and the image 510. The process of generating this registered image and the process of generating the difference image 530 are the same as the process of generating the registered image 240 and the process of generating the difference image 260 in embodiment 2. Therefore, a detailed description of the generation process will be omitted. The difference image 530 is also referred to as a third difference image.

[0103] The generation processing unit 140 generates an aligned image by aligning an object included in the image 500 with an object included in the image 510. The generation processing unit 140 simply generates a difference image 540 that shows the difference obtained by superimposing the aligned image and the image 510. The generation process of the aligned image and the generation process of the difference image 540 are the same as the generation process of the aligned image 50 and the generation process of the difference image 70 in the first embodiment. Therefore, a detailed description of the generation process will be omitted. The difference image 540 is also referred to as a second difference image.

[0104] The generation processing unit 140 generates a registered image by aligning an object included in the image 500 with an object included in the image 510. Furthermore, when performing this alignment, the generation processing unit 140 performs the alignment taking into account the area of ​​the image 510 that corresponds to the difference area indicated by the difference image 540. The generated registered image is also referred to as a third registered image. The generation processing unit 140 generates a difference image 550 that shows the difference obtained by simply superimposing the registered image and an image that has not been aligned. The process of generating this registered image and the process of generating the difference image 550 are the same as the process of generating the registered image 90 and the process of generating the difference image 92 in embodiment 3. Therefore, a detailed description of the generation process will be omitted. The difference image 550 is also referred to as a fourth difference image.

[0105] The generation processing unit 140 generates a reference image 560 that shows a common portion 561 among the difference image 520 , the difference image 530 , the difference image 540 , and the difference image 550 .

[0106] According to the sixth embodiment, the information processing device 100 can further improve the accuracy of detecting changes.

[0107] Seventh Embodiment Next, a seventh embodiment will be described. In the seventh embodiment, differences from the first to sixth embodiments will be mainly described. In the seventh embodiment, descriptions of the commonalities between the first to sixth embodiments will be omitted.

[0108] 10 is a block diagram showing the functions of the information processing device according to embodiment 7. The information processing device 100 further includes a selection unit 160.

[0109] A part or all of the selection unit 160 may be realized by a processing circuit. Alternatively, a part or all of the selection unit 160 may be realized as a program module executed by the processor 101. The function of the selection unit 160 will be described later.

[0110] Fig. 11 is a diagram showing a specific example of processing executed by the information processing device of embodiment 7. Fig. 11 shows images 600 and 610. Image 610 is an image generated after image 600. For example, image 600 is an image showing the state before work is performed on the board. Image 610 is an image showing the state after work is performed. Image 600 is also referred to as the first image. Image 610 is also referred to as the second image.

[0111] Image 600 includes object 601 and object 602. Image 610 includes object 611. Object 601 and object 611 are the same object. In this way, image 610 includes the same object as the object included in image 600. Images 600 and 610 are images in which the shapes and positions of the objects are different due to differences in the shooting directions, and in which different distortions have occurred due to compositing, wide-angle shooting, etc.

[0112] The generation processing unit 140 generates a plurality of deformed images based on the image 600 or the image 610. In other words, the generation processing unit 140 generates a plurality of deformed images based on either the image 600 or the image 610. The generation processing unit 140 generates a plurality of deformed images based on the image 600. The plurality of deformed images are deformed image 600a, deformed image 600b, etc.

[0113] When a deformed image is generated, multiple divided regions may be set in image 600, and movement may be performed for each divided region. Furthermore, when a deformed image is generated, an object included in image 600 may be moved in any direction. When a deformed image is generated, the amount of movement of an object included in image 600 may be any amount. For example, the position of object 601 included in deformed image 600a is the position where object 601 included in image 600 is moved to the right. The position of object 602 included in deformed image 600a is the position where object 602 included in image 600 is moved to the lower left.

[0114] Furthermore, when a deformed image is generated, the generation processing unit 140 may generate a plurality of deformed images by moving the entire image 600. For example, Fig. 11 shows an image 620 generated by moving the entire image 600.

[0115] In the following description, deformed image 600a, deformed image 600b, etc. will be used. The generation processing unit 140 generates multiple reference images showing the common location using each of the multiple deformed images and an unchanged image. Specifically, the generation processing unit 140 generates multiple reference images showing the common location using each of the deformed images 600a, 600b, etc. and image 610. The processing will be described in detail. The generation processing unit 140 generates a reference image showing the common location using deformed image 600a and image 610. The generation method is the same as that described in embodiment 1. As a result, the generation processing unit 140 generates an image such as reference image 80. Next, the generation processing unit 140 generates a reference image showing the common location using deformed image 600b and image 610. The generation method is the same as that described in embodiment 1. As a result, the generation processing unit 140 generates an image such as reference image 80. The generation processing unit 140 can similarly generate multiple reference images showing the common location by repeating the generation process.

[0116] The selection unit 160 selects, from among the plurality of reference images, the image with the smallest area of ​​common parts as the true reference image.

[0117] For example, the image processing unit 130 compares the true reference image with the image 600, and detects, as changed points, portions in the image 600 that correspond to common portions. Also, for example, the image processing unit 130 compares the true reference image with the image 610, and detects, as changed points, portions in the image 610 that correspond to common portions. In this way, the image processing unit 130 can detect changed points.

[0118] According to the seventh embodiment, the information processing device 100 can further improve the accuracy of detecting changes by generating a true reference image.

[0119] The above describes a case where multiple reference images showing a common location are generated by the generation method described in Embodiment 1. The generation processing unit 140 may generate multiple reference images showing a common location by the generation methods described in Embodiments 2 to 6.

[0120] The features of the above-described embodiments can be combined with each other as appropriate.

[0121] 10 Image, 11, 12 Object, 20 Image, 21 Object, 22 Frame, 23, 24, 25 Area, 30 State, 40 Difference image, 50 Aligned image, 60 State, 70 Difference image, 71, 72, 73 Difference area, 80 Reference image, 81 Common area, 90 Image, 91 State, 92 Difference image, 93 Reference image, 93a Common area, 100 Information processing device, 101 Processor, 102 Volatile storage device, 103 Non-volatile storage device, 110 Storage unit, 120 Acquisition unit, 130 Image processing unit, 140 Generation processing unit, 150 Output unit, 160 Selection unit, 200 Image, 201, 202 Object, 210 Image, 211 object, 212 region, 212a edge, 213 region, 213a edge, 214 region, 215 frame, 220 state, 230 difference image, 231, 232, 233 difference region, 233 difference region, 240 aligned image, 250 state, 260 difference image, 270 reference image, 271 common part, 300 image, 301, 302 object, 310 image, 311 object, 320 difference image, 330 difference image, 340 difference image, 350 reference image, 351 common part, 400 image, 401, 402 object, 410 image, 411 object, 420 difference image, 430 Difference image, 440 difference image, 450 reference image, 451 common location, 500 image, 501, 502 object, 510 image, 511 object, 520 difference image, 530 difference image, 540 difference image, 550 difference image, 560 reference image, 561 common location, 600 image, 600a deformed image, 600b deformed image, 601, 602 object, 610 image, 611 object, 620 image.

Claims

1. An information processing apparatus comprising: an acquisition unit that acquires a first image and a second image; and a generation processing unit that generates a first difference image indicating a difference obtained simply by overlapping the first image and the second image, generates a first alignment image based on the target image by aligning an object included in the target image, which is an image selected from among the first image and the second image, with an object included in the non-target image, which is an image not selected in the target image, generates a second difference image indicating a difference obtained simply by overlapping the first alignment image and the non-target image, and generates a reference image indicating a common portion between the first difference image and the second difference image.

2. The information processing apparatus according to claim 1, wherein when performing the alignment, the generation processing unit performs the alignment in consideration of an area of the non-target image corresponding to a difference area that is an area of the difference indicated by the first difference image.

3. The information processing apparatus according to claim 1, wherein when performing the alignment, the generation processing unit performs the alignment in consideration of a peripheral area of the area of the non-target image corresponding to a difference area that is an area of the difference indicated by the first difference image.

4. The information processing apparatus according to any one of claims 1 to 3, wherein after the first difference image is generated, the generation processing unit aligns an object included in the target image with an object included in the non-target image, and performs the alignment in consideration of an area of the non-target image corresponding to a difference area that is an area of the difference indicated by the first difference image, thereby generating a second alignment image based on the target image, generates a third difference image indicating a difference obtained simply by overlapping the second alignment image and the non-target image, and generates a reference image indicating a common portion among the first difference image, the second difference image, and the third difference image.

5. After the second difference image is generated, the generation processing unit aligns the objects included in the target image with the objects included in the non-target image, and performs the alignment in consideration of the region of the non-target image corresponding to the difference region that is the region of the difference indicated by the second difference image, thereby generating a third aligned image based on the target image. The generation processing unit generates a fourth difference image indicating the difference obtained simply by overlapping the third aligned image and the non-target image, and generates a reference image indicating the common part among the first difference image, the second difference image, the third difference image, and the fourth difference image. The information processing apparatus according to claim 4.

6. An acquisition unit that acquires a first image and a second image; a generation processing unit that generates a first aligned image based on the target image by aligning the objects included in the target image, which is the image selected from among the first image and the second image, with the objects included in the non-target image, which is the image not selected from the target image, generates a first difference image indicating the difference obtained simply by overlapping the first aligned image and the non-target image, generates a second aligned image based on the target image by aligning the objects included in the target image with the objects included in the non-target image and performing the alignment in consideration of the region of the non-target image or the peripheral region of the region of the non-target image corresponding to the difference region that is the region of the difference indicated by the first difference image, generates a second difference image indicating the difference obtained simply by overlapping the second aligned image and the non-target image, and generates a reference image indicating the common part between the first difference image and the second difference image. Each of the first image and the second image is an image in which the shape and position of an object are different due to, for example, different shooting directions, and is an image in which different distortions occur due to, for example, synthesis or wide-angle shooting. The information processing apparatus.

7. The generation processing unit generates a third difference image indicating the difference obtained simply by overlapping the first image and the second image, and generates a reference image indicating the common part among the first difference image, the second difference image, and the third difference image. The information processing apparatus according to claim 6.

8. Further comprising a selection unit, wherein the generation processing unit generates a plurality of deformed images based on the first image or the second image, generates a plurality of the reference images using each of the plurality of deformed images and the non-changed image, and the selection unit selects, as the true reference image, the image with the smallest area of the common part from among the plurality of reference images. The information processing apparatus according to any one of claims 1 to 7.

9. Further comprising an image processing unit that executes a process of comparing the reference image with the first image and detecting, in the first image, the part corresponding to the common part as a change point, or a process of comparing the reference image with the second image and detecting, in the second image, the part corresponding to the common part as a change point. The information processing apparatus according to any one of claims 1 to 8.

10. Further comprising an output unit, wherein the image processing unit marks the change point in the first image or the second image, and the output unit outputs the marked image. The information processing apparatus according to claim 9.

11. An information processing method, wherein an information processing apparatus acquires a first image and a second image, generates a first difference image showing a difference obtained simply by overlapping the first image and the second image, performs alignment of an object included in a target image, which is an image selected from among the first image and the second image, with an object included in a non-target image, which is an image not selected in the target image, to generate a first aligned image based on the target image, generates a second difference image showing a difference obtained simply by overlapping the first aligned image and the non-target image, and generates a reference image showing a common part between the first difference image and the second difference image, and each of the first image and the second image is an image in which the shape and position of an object are different due to, for example, different shooting directions, and is an image in which different distortions occur due to, for example, synthesis or wide-angle shooting.

12. An information processing program for causing an information processing apparatus to execute a process of acquiring a first image and a second image, generating a first difference image showing a difference obtained simply by overlapping the first image and the second image, generating a first alignment image based on the target image by aligning an object included in the target image, which is an image selected from among the first image and the second image, with an object included in the non-target image, which is an image not selected in the target image, generating a second difference image showing a difference obtained simply by overlapping the first alignment image and the non-target image, and generating a reference image showing a common portion between the first difference image and the second difference image, wherein each of the first image and the second image is an image in which the shape and position of an object are different due to, for example, different shooting directions, and is an image in which different distortions have occurred due to, for example, synthesis or wide-angle shooting.

Images