Program, information processing device, and determination method

Abstract

To provide a program, an information processing apparatus, and a determination method for determining the identity of electronic data including a variable part.SOLUTION: An information processing apparatus 10 comprises: a variable part comparison unit 17 that detects a non-matching part between a variable part in a first image to be a reference and a variable part in a second image to be compared based on information on the variable part in the first image and the variable part in the second image; and an output unit 16 that outputs the non-matching part detected by the variable part comparison unit 17. The variable part comparison unit 17 changes a method for detecting the non-matching part based on the information on the variable parts.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing device, and a determination method. [Background technology]

[0002] There are times when you want to determine whether two images are the same. For example, in fields such as commercial printing, where high-quality printouts are required, users may want to evaluate whether printouts with the same image quality can be obtained even if image processing software such as printer drivers is updated. Also, when users compress images, they may want to determine whether the images can be considered the same even if the image quality is reduced by compression.

[0003] As a technique for comparing images, a technique for evaluating the user's quality of experience during video distribution is known (see, for example, Patent Document 1). Patent Document 1 discloses a method for acquiring at least one of a quality degradation index of input transcoded video and a bit rate of the input transcoded video and a bit rate of the video before transcoding, and estimating the user's quality of experience of the transcoded video based on the acquired values. Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques have the problem of making it difficult to determine the identity of electronic data that includes variable parts. For example, when a web page or the like contains variable parts such as animation, it is difficult to determine the identity because the animation part changes depending on the timing of outputting (capturing) a still image for comparison.

[0005] SUMMARY OF THE INVENTION In view of the above-mentioned problems, an object of the present invention is to provide an information processing device that can determine the identity of electronic data that includes a variable portion. [Means for solving the problem]

[0006] In view of the above-mentioned problems, the present invention provides an information processing device that functions as a variable part comparison unit that detects mismatched parts of variable parts that a first image serving as a reference and a second image to be compared have, based on information about variable parts that the first image and the second image to be compared have, and an output unit that outputs the mismatched parts detected by the variable part comparison unit, and the variable part comparison unit changes a detection method for the mismatched parts based on the information about the variable parts, When the information about the variable part includes information that the variable part changes linearly, an image at the start of the change, and an image at the end of the change, the variable part comparison part determines whether or not the image of the variable part of the second image matches an image during the change from the image at the start of the change to the image at the end of the change. Characterized by Offering a program . [Effects of the Invention]

[0007] It is possible to provide an information processing device that can determine the identity of electronic data that includes a variable portion. [Brief explanation of the drawings]

[0008] [Figure 1] 10 is a diagram outlining a method for determining whether electronic data including a variable portion is identical. FIG. [Figure 2] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device or an information processing system that performs identity determination. [Figure 3] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device or a server. [Figure 4] FIG. 1 is an example of a functional block diagram illustrating functions of an information processing device divided into blocks. [Figure 5] FIG. 10 is an example of a functional block diagram illustrating functions of a variable section comparison section. [Figure 6] 10A and 10B are diagrams illustrating an example of the arrangement of variable parts when the electronic data is a layout image of an operation panel. [Figure 7] FIG. 10 is a diagram showing an example of the arrangement of variable parts when electronic data is a Web page. [Figure 8] 10A and 10B are diagrams illustrating a comparison method when variable part information is linear. [Figure 9] 10A and 10B are diagrams illustrating the CMYK ratios of the colors of each pixel during the change. [Figure 10]10A and 10B are diagrams illustrating a comparison method when variable part information is a pattern. [Figure 11] FIG. 10 is a diagram illustrating a comparison method when variable part information is a date and time. [Figure 12] 10A to 10C are diagrams illustrating a method for generating difference information of a CMYK image by a difference information generating unit for a still image. [Figure 13] 10A and 10B are diagrams illustrating a method for detecting continuous differences that are at least large enough to be recognized by the human eye. [Figure 14] 10A and 10B are diagrams illustrating a method for detecting differences that exist within a certain region at a rate that is greater than or equal to the rate at which the human eye can perceive them. [Figure 15] FIG. 10 is a diagram illustrating a method for generating difference information. [Figure 16] 10A and 10B are examples of difference information between comparison images 1 and 2 and an enlarged view of the difference information. [Figure 17] FIG. 10 is a diagram illustrating a method for detecting a layout pattern from the layout of differences. [Figure 18] 10 is an example of a flowchart illustrating a procedure in which a variable portion comparison unit compares a base image with a comparison image 2 for a variable portion of electronic data. [Figure 19] FIG. 10 is a diagram illustrating an example of a mismatched part display screen displayed by the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing device and a determination method performed by the information processing device will be described below as an example of an embodiment of the present invention with reference to the accompanying drawings.

[0010] <Outline of identity determination> Figure 1 is a diagram outlining a method for determining whether electronic data containing a variable portion is identical. First, there are various types of variable portion, but in Figure 1, it is assumed to be animation. Animations can change linearly or change based on a pattern.

[0011] Figure 1(a) shows an animation that changes linearly, while Figure 1(b) shows an animation that changes based on a pattern. A linear change refers to a change that can be interpolated between the color at the start of the change and the color at the end of the change. In the animation of Figure 1(a), from left to right, the color around the square in the center gradually becomes the same as the center. The information processing device can determine the surrounding colors during the change by interpolation.

[0012] Therefore, if an information processing device has information that two images change linearly at different times (for example, when the change starts and when the change ends), it can interpolate the color of each pixel during the change and compare it with the image being compared.

[0013] A pattern-based change is a change that expresses animation by switching between multiple still images in a short period of time. Figure 1(b) shows four still images of squares with different colors, but the fact that they are squares and their number are merely an example for the convenience of explanation. Another example of a pattern-based change is a change according to the date and time.

[0014] If the information processing device has four still images and information that the images change based on a pattern, it can compare each of the four images with the comparison target image in a brute-force manner.

[0015] In this way, the information processing apparatus of this embodiment changes the comparison method depending on whether the variable area is linear, pattern, or date and time, and therefore can appropriately detect mismatches in the variable portion.

[0016] Although there is a method in which the information processing device masks (ignores) the animation portion for comparison, this method cannot detect that the masked portion is not identical. In this embodiment, such a problem can also be prevented.

[0017] <Terminology> In this embodiment, "identical" refers to only differences that are not perceptible to the human eye, and "non-identical" refers to differences that are perceptible to the human eye. Therefore, "identical" may be referred to as "similar," and "non-identical" may be referred to as "dissimilar."

[0018] The information about the variable part is information about how the variable part changes. In this embodiment, this is explained using the term variable part information, and has three aspects: linear, pattern, and date and time.

[0019] A variable part is a part that is not a still image. The variable part can change in any way. The variable part can be created by animation, switching between still images, video (live action), etc. Images in the variable part are sometimes collectively called a video.

[0020] A mismatch is a location where the two images do not match, and can be on a pixel-by-pixel basis or a block of pixels.

[0021] <Configuration example> FIG. 2 is a configuration diagram of an example of an information processing device 10 or an information processing system 100 that performs identity determination. In this embodiment, there is a system configuration (FIG. 2(a)) in which the information processing device 10 performs identity determination, and a system configuration (FIG. 2(b)) in which the server 30 performs identity determination. In the system configuration of FIG. 2(a), the information processing device 10 acquires two pieces of electronic data stored in a storage unit and information used for comparison (variable part information (linear, pattern, date and time), basic image, and position information), which will be described later, to perform identity determination, and outputs the determination result on a display. The determination result may be sent by email or may be stored in a storage unit. The determination result may also be saved in the cloud.

[0022] The information processing device 10 may be, for example, a PC (Personal Computer), a tablet terminal, a PDA, a smartphone, or any other device on which software can run.

[0023] In the information processing system 100 of Fig. 2(b), the information processing device 10 and the server 30 can communicate with each other via a network N. A server is a device that mainly performs information processing on a network and responds with processing results in response to a request received via the network.

[0024] In the system configuration of Fig. 2(b), the information processing device 10 transmits two or more images to the server 30. The server 30 determines whether the images are identical and transmits the determination result to the information processing device 10. The determination result may be transmitted by email or may be stored in the cloud.

[0025] In the configuration of FIG. 2(b), the server 30 functions as a so-called web server, generating screen information for a portal screen that accepts image data (written in HTML, XML, scripting language, CSS (cascading style sheet), etc., and mainly analyzed and displayed by browser software) and providing it to the information processing device 10. A web browser operates in the information processing device 10, which receives the screen information and displays a web page. This web page has an interface that allows the information processing device 10 to register two pieces of electronic data and information used for comparison (variable part information (linearity, pattern, date and time), basic image, and position information), which will be described later; the user registers this information on the web page and sends it to the server 30.

[0026] Web pages may be provided by web applications. A web application is software or a mechanism executed on a browser that operates through the cooperation of a program written in a programming language (e.g., JavaScript (registered trademark)) that runs on a browser and a program on the web server. By using a web application, an information processing device can dynamically change web pages.

[0027] <Hardware configuration example> 3 is a diagram showing the hardware configuration of the information processing device 10 or the server 30. Here, the hardware configuration of the information processing device 10 will be described.

[0028] As shown in FIG. 3, the information processing device 10 is constructed by a computer, and as shown in FIG. 3, it includes a CPU 501, a ROM 502, a RAM 503, an HD 504, an HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, an optical drive 514, and a media I / F 516.

[0029] Of these, the CPU 501 controls the overall operation of the information processing device 10. The ROM 502 stores programs, such as an IPL, used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 under the control of the CPU 501. The display 506 displays various information, such as a cursor, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external devices are, for example, USB (Universal Serial Bus) memories, printers, etc. The network I / F 509 is an interface for data communication using a communication network. The bus line 510 is an address bus, a data bus, etc. for electrically connecting the components, such as the CPU 501, shown in FIG. 3.

[0030] The keyboard 511 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The optical drive 514 controls reading and writing of various data from a removable optical storage medium 513. The optical storage medium 513 may be a CD, DVD, Blu-Ray (registered trademark), etc. The media I / F 516 controls reading and writing (storing) of data from a storage medium 515 such as a flash memory.

[0031] <About the function> Fig. 4 is an example of a functional block diagram that explains the functions of the information processing device 10 by dividing them into blocks. In Fig. 4, unless otherwise specified, the explanation is based on the functions of the information processing device 10, but the server 30 may have the same functions as the information processing device 10, although it differs in that it further has a communication unit.

[0032] The information processing device 10 has an image acquisition unit 11, a difference information generation unit 12, a difference information output unit 13, a difference information arrangement recognition unit 14, a determination unit 15, an output unit 16, and a variable part comparison unit 17. Each of these functional units of the information processing device 10 is a function or means realized when any of the components shown in Fig. 3 operates in response to an instruction from the CPU 501 in accordance with a program loaded from the HD 504 to the RAM 503.

[0033] The image acquisition unit 11 acquires two pieces of electronic data to be compared, as well as information used for the comparison (variable part information (linear, pattern, date and time), basic image, and position information). Each of the two pieces of electronic data is a still image, but is expected to look identical at the pixel level when viewed by a human. For example, a user may convert the same print object into a still image using a printer driver or RIP software, and the image acquisition unit 11 acquires each of these still images. Print objects vary, but in this embodiment, it is assumed that there will be variable parts.

[0034] The variable part comparison unit 17 detects mismatched parts (difference information) in the variable part of the electronic data, and stores the mismatched parts in the difference information storage unit 19. Details will be described later.

[0035] The difference information generating unit 12 generates difference information between two images of interest for the still parts (other than the variable parts) of the electronic data. For example, the difference information is generated by determining whether or not there are identical pixels (CMYK) for each pixel position determined by the image resolution. Note that the difference information generating unit 12 also uses position information indicating where the variable parts are to identify the still parts.

[0036] The difference information output unit 13 stores the difference information generated by the difference information generation unit 12 in the difference information storage unit 19. The difference information is two-dimensional information with the same resolution (size) as the image.

[0037] The difference information arrangement recognition unit 14 recognizes the arrangement of the differences. As will be described in detail later, it detects continuous differences, proportions within a certain area, and predetermined patterns from the difference information.

[0038] Based on the difference arrangement recognized by the difference information arrangement recognition unit 14, the judgment unit 15 judges whether there is a difference arrangement visible to humans (a difference arrangement larger than the threshold at which the human eye can determine that there is a difference; the threshold is determined based on whether the human eye can determine that there is a difference between the two images). If there is a visible difference, it judges that the two images are not identical, and if there is no visible difference, it judges that the two images are identical. In other words, it judges whether the consecutive differences and the proportion within a certain area are greater than or equal to the threshold at which the human eye can determine that there is a difference. If a predetermined pattern is detected, the judgment unit 15 judges that the images are not identical. Note that these three judgment criteria may be used in conjunction with two or more ANDs or with an OR.

[0039] The output unit 16 highlights the layout of the differences and the total amount of intra-region differences and displays them on the display 506 or prints them out.

[0040] Next, the variable section comparison section 17 will be described in detail with reference to Fig. 5. Fig. 5 is a functional block diagram illustrating the functions of the variable section comparison section 17.

[0041] The variable part comparison unit 17 has a variable part information storage unit 21, a comparison image 1 storage unit 22, a comparison image 2 storage unit 23, and an image comparison unit 24.

[0042] The comparison image 1 storage unit 22 is a storage means for storing the comparison image 1 (an example of a first image). The comparison image 1 storage unit 22 may be located on a network accessible by the information processing device 10. The comparison image 1 is, for example, the entire electronic data output by reference software, such as an image output by a printer driver or RIP software prior to an upgrade. The comparison image 1 is an image captured at a certain moment in electronic data having a variable portion. Therefore, the state of the variable portion is unknown.

[0043] The comparison image 2 storage unit 23 is a storage means that stores the comparison image 2 (an example of a second image). The comparison image 2 storage unit 23 may be located on a network accessible by the information processing device 10. The comparison image 2 is, for example, the entire electronic data output by new software, and is an image (image to be compared) that is to be determined as being identical to the comparison image 1. The comparison image 2 is, for example, an image output by an upgraded printer driver or RIP software. The comparison image 2 is an image captured at a certain moment in electronic data that has a variable portion. Therefore, the state of the variable portion is unknown.

[0044] The variable part information storage unit 21 stores variable part information (linear, pattern, date and time) about how the variable part of the comparison image 1 changes, as well as basic images and position information. The variable part information indicates how the variable part changes (linear, pattern, date and time). If the variable part information is linear, the basic images are the image at the start of the change and the image at the end of the change. If the variable part information is pattern, the basic images are all the images that make up the animation. If the variable part information is time, the basic images are number images from 0 to 9. The basic images are the source of the variable part of the comparison image 1. The position information is the coordinates of the variable part in the comparison image 1.

[0045] The image comparison unit 24 compares the comparison image 1 and the comparison image 2 using the variable part information and position information stored in the variable part information accumulation unit 21. The image comparison unit 24 determines whether there is a mismatch for each pixel. As will be described later, the image comparison unit 24 compares the base image and the comparison image 2, but since the base image is the image that serves as the basis for the comparison image 1, this is essentially the same as comparing the comparison images 1 and 2.

[0046] The difference information placement recognition unit 14 recognizes the placement of difference information based on the mismatched parts. This allows the determination of whether two images can be considered identical based on whether they are "judged differently by human vision" even in the variable parts. However, the determination unit 15 may also determine whether they are completely identical.

[0047] <Example of electronic data to be compared> Electronic data having variable parts will be described with reference to Figs. 6 and 7. Fig. 6 shows an example of the arrangement of variable parts when the electronic data is a layout image of an operation panel. An operation panel has areas where the color of a selected button changes, or the size changes to prompt the user to perform an operation. The date and time may also change. In the layout image of an operation panel, these can be variable parts.

[0048] For example, in Fig. 6, the color of two buttons 201 and 202 changes when pressed. Also, the start button 203 repeatedly gets larger and smaller little by little to prompt the user to operate it. Also, the date and time 204 changes over time.

[0049] The manufacturer of the operation panel already knows the variable section information (linear, pattern, date and time), basic image, and position information for the operation panel. For example, the variable section information for buttons 201 and 202 is a pattern, the variable section information for start button 203 is a linear, and the variable section information for date and time 204 is date and time. The manufacturer of the operation panel can also prepare basic images for buttons 201 and 202, start button 203, and date and time 204. The manufacturer of the operation panel stores the variable section information (linear, pattern, date and time), basic image, and position information in variable section information storage unit 21.

[0050] 7 shows an example of the arrangement of variable sections when the electronic data is a web page. A variable section 210 is arranged in the upper right corner of this web page. The variable section 210 may be an animation or a video (live action). Therefore, the image of the variable section 210 changes over time.

[0051] A web page developer or the like already knows the variable part information (linearity, pattern, date and time), basic image, and position information for the web page. The web page developer or the like can prepare a basic image for the variable part 210. The web page developer or the like stores the variable part information (linearity, pattern, date and time), basic image, and position information in the variable part information storage unit 21.

[0052] <Comparative example of variable section> Next, a method for comparing variable parts in electronic data will be described with reference to FIGS.

[0053] <<When the variable part information is linear>> First, Figure 8 is a diagram explaining the comparison method when the variable part information is linear. When the variable part information is linear, images at the start and end of the change are obtained as basic images. Figure 8(a) is the image at the start of the change, and Figure 8(b) is the image at the end of the change. The dotted line in Figure 8(a) indicates the outer edge of the image at the start of the change and does not actually exist. As shown in Figures 8(a) and (b), the image at the start of the change and the image at the end of the change are the same size.

[0054] For ease of explanation, in Figure 8(a), the image at the start of the change has a colored area 221 and a white area 222. The white area 222 gradually becomes colored, eventually becoming the image at the end of the change. A linear change means that, for each pixel, the pixel values ​​(CMYK) of the image at the start of the change change uniformly to the pixel values ​​at the end of the change. The pixel values ​​correspond to colors.

[0055] As shown in FIG. 1(a), comparison images 1 and 2 are images captured at a certain moment during the transition from the image at the start of the transition to the image at the end of the transition, and it is highly unlikely that comparison images 1 and 2 will be captured at the exact same moment. However, because it is assumed that the color of each pixel changes linearly, there are rules regarding the CMYK ratios of the color of each pixel during the transition. In other words, if a certain timing during the transition is identified, the CMYK ratios of the color of each pixel during the transition will also be identified. Therefore, the variable part comparison unit 17 determines whether the CMYK ratios of comparison image 2 match the CMYK ratios at any timing between the start of the transition and the end of the transition. This allows the variable part comparison unit 17 to determine whether the image in the variable part of comparison image 2 matches the image during the transition from the image at the start of the transition to the image at the end of the transition.

[0056] Fig. 9 is a diagram illustrating the CMYK ratios of the color of each pixel during the change. For the sake of explanation, Fig. 9 illustrates the case of a change from white (0,0,0,0) to green (100,0,100,0) (change in white area 222). Fig. 9(a) shows the change in the C ratio, Fig. 9(b) shows the change in the M ratio, Fig. 9(c) shows the change in the Y ratio, and Fig. 9(d) shows the change in the K ratio.

[0057] First, in the case of white, the CMYK ratio at the end of the change is calculated using the following formula: C ratio = 0 / (0+0+0+0) = 0.0% M ratio = 0 / (0+0+0+0) = 0.0% Y ratio = 0 / (0+0+0+0) = 0.0% K ratio = 0 / (0+0+0+0) = 0.0% In the case of green, the CMYK ratio at the end of the change is calculated using the following formula: C ratio = 100 / (100+0+100+0) = 50% M ratio = 0 / (100+0+100+0) = 0% Y ratio = 100 / (100+0+100+0) = 50% K ratio = 0 / (100+0+100+0) = 0% Since the pixel values ​​of CMYK change linearly, the variable section comparison section 17 simply connects C together, M together, Y together, and K together at the start and end of the change with straight lines.

[0058] Assume that the CMYK ratios of a pixel in comparison image 2 are A%, B%, C%, and D% = A:B:C:D, respectively. The variable part comparison unit 17, for example, identifies the timing Tm at which C takes on A% on the X axis in FIG. 9(a), and obtains the M, Y, and K ratios b, c, and d at this timing from FIGS. 9(b), (c), and (d). The variable part comparison unit 17 determines whether the obtained M ratio b matches B, whether the obtained Y ratio c matches C, and whether the obtained K ratio d matches D. In this way, the variable part comparison unit 17 can compare the images of the variable parts.

[0059] The variable section comparison unit 17 may identify the timing when M takes B% on the X-axis of Figure 9(b) and obtain the ratios a, c, and d of C, Y, and K at this timing from Figures 9(a), (c), and (d). The variable section comparison unit 17 may identify the timing when Y takes C% on the X-axis of Figure 9(c) and obtain the ratios a, b, and d of C, M, and K at this timing from Figures 9(a), (b), and (d). The variable section comparison unit 17 may identify the timing when K takes D% on the X-axis of Figure 9(d) and obtain the ratios a, b, and c of C, M, and Y at this timing from Figures 9(a), (b), and (c).

[0060] In this way, the variable part comparison unit 17 can determine whether the comparison image 2 is identical to the base image by generating CMYK pixel information that changes linearly based on the base image. If the CMYK ratios do not match, the variable part comparison unit 17 records the fact that there is a mismatch in association with coordinates (which may be coordinates in the variable part or coordinates in the electronic data) and saves this in the difference information storage unit 19. If there is a mismatch, the variable part comparison unit 17 may also record the extent of the difference in association with the coordinates.

[0061] <<If the variable part information is a pattern>> Next, Fig. 10 is a diagram illustrating a comparison method when the variable portion information is a pattern. When the variable portion information is a pattern, images 232 to 235 of all patterns are obtained as basic images. Fig. 10(a) shows images 232 to 235 of all patterns of the variable portion. In the figure, the pattern is made up of four images 232 to 235 of different colors, but this is just an example. Also, in Fig. 10(a), only the colors are different, but there is no problem if the shapes change.

[0062] FIG. 10(b) shows the variable part of comparison image 2 (reference numeral 231). The size of the variable part of comparison image 2 is the same as that of all pattern images 232 to 235. The variable part of comparison image 2 is expected to match any of all pattern images 232 to 235 of the variable part. Therefore, the variable part comparison unit 17 can compare comparison image 2 with all pattern images 232 to 235 of the variable part in a brute force manner.

[0063] Note that the variable portion comparison unit 17 can end the comparison at that point if there is an image that matches the comparison image 2 in all of the variable portion pattern images 232 to 235. To reduce the number of comparisons, the variable portion comparison unit 17 may narrow down the comparison to images that have similar circumscribing rectangles or color histograms to the comparison image 2, and then perform the comparison.

[0064] The variable part comparison unit 17 compares each pixel of the comparison image 2 with the images 232 to 235 of the variable part pattern, with a maximum of one matching. The variable part comparison unit 17 records a mismatch for each of the images 232 to 235 in the pattern, associating it with coordinates. The variable part comparison unit 17 then identifies the image in the pattern with the fewest mismatched pixels (image 232 in the figure) and stores the correspondence between the coordinates (which may be the base image or comparison image 2) and the mismatch in the difference information storage unit 19. If there is a mismatch, the variable part comparison unit 17 may record the extent of the difference, associating it with the coordinates. If the pattern is given as a video, the variable part comparison unit 17 compares each frame that makes up the video.

[0065] <<If the variable part information is a date and time>> 11 is a diagram for explaining a comparison method when the variable portion information is a date and time. When the variable portion information is a date and time, a number image 241 of 0 to 9 is obtained as a basic image. FIG. 11(a) shows the number image 241 of 0 to 9.

[0066] FIG. 11(b) shows the variable part of comparison image 2. When the variable part information is date and time, the position information of the variable part is given for each number image 242 to 245. That is, in FIG. 11(b), the "1" image 242, the "8" image 243, the "3" image 244, and the "0" image 245 are each considered to be one variable part. Therefore, the variable part comparison unit 17 can compare each of the number images "1," "8," "3," and "0" 242 to 245 of comparison image 2 with the number images 0 to 9 241 in a brute force manner.

[0067] If the images 242 to 245 of the numbers "1," "8," "3," and "0" in the comparison image 2 match the images 241 of the numbers 0 to 9, the variable part comparison unit 17 can end the comparison at that point.

[0068] The variable part comparison unit 17 compares each pixel of the images 242 to 245 of the numbers "1," "8," "3," and "0" with the image 241 of the numbers 0 to 9, but there is only one match at most. The variable part comparison unit 17 records a mismatch for each of the images 242 to 245 of the numbers "1," "8," "3," and "0" in association with coordinates. The variable part comparison unit 17 then identifies the number image with the fewest mismatched pixels and stores the correspondence between the coordinates (which may be the base image or the comparison image 2) and the mismatch in the difference information storage unit 19. If there is a mismatch, the variable part comparison unit 17 may record the extent of the difference in association with the coordinates.

[0069] The variable part comparison unit 17 may also determine whether the number with the fewest mismatched pixels matches the correct date or time value. The variable part comparison unit 17 can identify the number with the fewest mismatched pixels, and then determines whether these numbers are appropriate as a date and time. For example, if the variable part is a date, then in YYYY / MM / DD, the correct values ​​for YYYY are 2000 to 2999, for MM 01 to 12, and for DD 01 to 31. If the variable part is a time, then in hh:mm, the correct values ​​for hh are 00 to 23, and for mm 00 to 59.

[0070] The variable part comparison unit 17 determines whether the number with the fewest mismatched pixels is the correct value based on the information that the variable part is the date or time, and stores the result in the difference information storage unit 19.

[0071] <Generating difference information for still images> FIG. 12 is a diagram for explaining a method of generating difference information of a CMYK image by the difference information generation unit 12 for a still image. A part of two images to be compared is enlarged and shown in FIG. 12. The difference information generation unit 12 determines whether there is a difference between the two images for each pixel position determined according to the resolution of the image (for example, in the case of an image with a resolution of 1980×1280, the vertical is equally divided into 1980 and the horizontal is equally divided into 1280 positions). In FIG. 12, the intersection points of the mesh are the pixel positions.

[0072] If pixels of the same color exist at the same pixel position in the two images, it is determined that there is no difference, and if there are no pixels of the same color, it is determined that there is a difference. The difference information generation unit 12 sets a flag at the pixel position where there is a difference. When the difference information is represented by an image, a dot is displayed at the pixel position where there is a flag. The difference information generation unit 12 prepares, for example, a table for a fraction of the number of pixel positions and records "presence or absence of a flag".

[0073] Alternatively, the difference information generation unit 12 may use a value corresponding to the magnitude of the difference as the difference information. For example, assume that there is the following difference at a certain pixel position between one image data and the other image data. A. With C and with K B. With C and with Y C. With C and nothing D. With C and with M In this case, it is considered that the differences are in the order of A < B < C < D. Therefore, the difference information generation unit 12 sets larger values in the above table in the order of A < B < C < D.

[0074] Also, the difference information generation unit 12 may calculate feature amounts by combining each pixel of an individual image with other pixels and then compare them. For example, pixels around the pixel position are weighted with a Gaussian filter or the like, and the difference between the weighted values is recorded in a table, or if there is a difference greater than a threshold value, a flag indicating that there is a difference is recorded in the table.

[0075] <Arrangement of differences> The following describes a method for determining whether the mismatched arrangement is humanly visible. The mismatched variable portions are hereinafter referred to as "differences."

[0076] First, let us explain how to recognize the placement of differences. (i) The pixels that are judged to be different are continuous and large enough to be recognized by the human eye. This article explains:

[0077] The difference information arrangement recognition unit 14 sequentially scans the pixel positions to detect consecutive differences.

[0078] 13A and 13B are diagrams illustrating a method for detecting consecutive differences that are large enough to be recognized by the human eye. In Fig. 13A, the difference information layout recognition unit 14 scans the pixel positions horizontally. This allows for the detection of consecutive differences 130 that are large enough to be recognized by the human eye.

[0079] 13(b), the difference information layout recognition unit 14 scans the pixel positions in the vertical direction, thereby detecting any differences 131 that are vertically continuous and of a size that can be recognized by the human eye.

[0080] 13, the arrangement of rectangular differences is detected by scanning both vertically and horizontally. The difference information arrangement recognition unit 14 detects the length of consecutive differences in the vertical or horizontal direction. It is also advisable for the difference information arrangement recognition unit 14 to scan diagonally as well.

[0081] The determination unit 15 then compares the length of the consecutive differences with a predetermined length (threshold value) and records that fact at the pixel positions of the consecutive differences that are longer than the predetermined length. The difference information arrangement recognition unit 14, for example, prepares a table for each pixel position and records "differences present" at all pixel positions of consecutive differences that are longer than the predetermined length.

[0082] This allows the output unit 16 of the information processing device 10 to highlight and display these pixels, allowing the user to recognize the arrangement of the differences at a glance.

[0083] In this embodiment, the predetermined length (threshold) can be set to a constant value regardless of the image.

[0084] Next, (ii) The pixels that are judged to be different exist in a certain area at a rate that is noticeable to the human eye. This article explains:

[0085] In this case, the difference data layout recognition unit 14 moves a window surrounding the pixel position in the difference data in order and determines whether or not there is a difference within this window that is larger than the percentage that can be recognized by the human eye. The size of this window corresponds to a certain area.

[0086] Figure 14 is a diagram illustrating a method for detecting differences that exist within a certain area at a rate that is noticeable to the human eye. A 4x4 pixel window 140 is shown in Figure 14, but the size of the window 140 is an example. The window 140 moves the pixel position at the center of the window 140 from the upper left corner to the right by one pixel at a time, and when it reaches the right edge, it moves back one pixel to the left edge. This movement is repeated until the pixel position at the center reaches the lower right corner.

[0087] A 4x4 pixel window 140 contains 5x5 = 25 pixel positions. The difference information layout recognition unit 14 counts how many pixel positions within the window are determined to have differences. For ease of explanation, in Figure 14, 10 pixel positions are determined to have differences.

[0088] The judgment unit 15 judges whether the "predetermined ratio of 10 / 25" is satisfied, and if so, records that fact at all pixel positions within the window in a table equal to the number of pixel positions. This allows the output unit 16 of the information processing device 10 to highlight and display those pixels, allowing the user to recognize the placement of the differences at a glance.

[0089] In this embodiment, the predetermined ratio (threshold value) can be set to a constant value regardless of the image.

[0090] Next, (iii) The pixels determined to be different exist in a specified arrangement pattern; This article explains:

[0091] In this case, the differential information layout recognition unit 14 holds the specified layout pattern. The differential information layout recognition unit 14 performs pattern matching on the differential information with the layout pattern and determines whether there is a differential layout that matches the layout pattern.

[0092] Fig. 15 is a diagram illustrating a method for generating difference information. Fig. 15(a) shows an example of comparison image 1, and Fig. 15(b) shows an example of comparison image 2. For the purpose of explanation, Fig. 15 shows comparison images 1 and 2, which are clearly different, but in reality, it may be difficult to tell the difference between comparison images 1 and 2 just by looking at them.

[0093] Fig. 16(a) shows the difference information between comparison images 1 and 2 in Fig. 15, and Fig. 16(b) is an enlarged view of the difference information. Fig. 16(c) shows an arrangement pattern 150. The arrangement pattern 150 is generated in advance as a pattern of differences that are easily perceived by the human eye. The arrangement pattern 150 in Fig. 16(c) is just one example, and may be strip-shaped, circular, geometric, or the like.

[0094] If the difference such as the enlarged view of FIG. 16(b) matches the layout pattern 150, the difference information layout recognition unit 14 determines that there is a difference, and two images that humans perceive as having a difference can be detected.

[0095] FIG. 17 is a diagram illustrating a method for detecting an arrangement pattern from a difference arrangement 160. For the sake of explanation, FIG. 17 shows a difference arrangement 160 having the same pattern as arrangement pattern 150. The difference information arrangement recognition unit 14 moves the arrangement pattern 150 one pixel at a time from the upper left corner of the difference information 151 to the right, and when it reaches the right end, it moves back one pixel to the left end. The difference information arrangement recognition unit 14 repeats this movement until it reaches the lower right corner.

[0096] The difference information arrangement recognition unit 14 determines whether or not differences are recorded in the difference information at the same positions as the pixels of the arrangement pattern 150, and if differences are recorded at all pixel positions, it determines that the arrangement pattern 150 has been detected from the difference information 151. Alternatively, if differences are recorded at the positions of a certain number or more (a certain percentage or more), it determines that the arrangement pattern 150 has been detected from the difference information 151.

[0097] In FIG. 17, a difference arrangement 160 that matches the arrangement pattern 150 is located at the lower right of the difference information 151, and the difference information arrangement recognition unit 14 can detect this difference arrangement 160.

[0098] The difference information layout recognition unit 14 records, in a table equal to the number of pixel positions, the pixel positions that match the layout pattern 150 to that effect. This allows the output unit 16 of the information processing device 10 to highlight and display these pixels, allowing the user to recognize the layout of the differences at a glance.

[0099] <Comparison of variable parts> 18 is a flowchart showing the procedure by which the variable part comparison unit 17 compares the base image with the comparison image 2 for the variable part of the electronic data. The processing in FIG. 18 starts when the variable part of the electronic data is compared.

[0100] First, the variable part comparison unit 17 acquires the variable part information, the basic image, and the position information from the variable part information storage unit 21, and acquires the comparison image 2 from the comparison image 2 storage unit 23 (S1).

[0101] Next, the variable part comparison unit 17 determines whether the variable part information is linear, pattern, or date and time (S2).

[0102] If the variable part information is linear, the variable part comparison unit 17 creates the CMYK ratios from the images at the start and end of the change, as shown in FIG. 10 (S3).

[0103] Then, the variable part comparison unit 17 records for each pixel whether the CMYK ratio of the image in the variable part of the comparison image 2 matches the ratio at any timing from the start of the change to the end of the change in the basic image (S4).

[0104] If the variable part information is a pattern, the variable part comparison unit 17 compares the image of the variable part of the comparison image 2 with all the images that make up the pattern, and identifies the image with the least difference (S5).

[0105] Then, the variable part comparison unit 17 records the mismatched pixels in the image with the smallest difference in association with their coordinates (S6).

[0106] If the variable part information is date and time, the variable part comparison unit 17 compares the image of the variable part of the comparative image 2 with the image of numbers 0 to 9, and identifies the number image with the least difference (S7). Note that the comparative image 2 is divided into single digits.

[0107] Then, the variable portion comparison unit 17 records the mismatched pixels in the number image or variable portion image with the smallest difference in association with the coordinates (S8).

[0108] The variable part comparison unit 17 also records whether the number image with the least difference satisfies the correct value of the date or time (S9).

[0109] <Example of mismatched parts> FIG. 19 is an example of a mismatch area display screen displayed by the information processing device 10. The mismatch area display screen has a comparison image 1 column 251, a comparison image 2 column 252, a mismatch area display column 253, and an explanation column 254. Comparison image 1 is displayed in the comparison image 1 column 251, and comparison image 2 is displayed in the comparison image 2 column 252. Pixels determined to be mismatched are displayed with marks 255 in the mismatch area display column 253. This allows the user to easily understand which parts have been determined to be mismatched and how many mismatched pixels there are. Furthermore, the explanation column 254 displays the number of mismatched pixels and the result of the ambiguous judgment. The ambiguous judgment is the result of a judgment as to whether the mismatched arrangement is visible to the human eye. Therefore, the user can comprehensively determine whether the two images are mismatched by referring to the number of mismatched pixels and the result of the ambiguous judgment.

[0110] <Major Effects> As described above, the information processing apparatus of this embodiment changes the comparison method depending on whether the changing area is linear, pattern, or date and time, and therefore can appropriately detect differences in the variable portion.

[0111] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0112] For example, the configuration example shown in FIG. 4 is divided according to main functions to facilitate understanding of the processing by the information processing device 10. The method of dividing the processing units and their names do not limit the present invention. The processing of the information processing device 10 can be divided into even more processing units depending on the processing content. Furthermore, it can also be divided so that one processing unit includes even more processes.

[0113] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions. [Explanation of symbols]

[0114] 10. Information processing equipment 30 servers 100 Information Processing Systems [Prior art documents] [Patent documents]

[0115] [Patent Document 1] JP 2016-46685 A

Claims

1. An information processing device a variable part comparison unit that detects mismatches between the variable parts of a first image serving as a reference and a second image to be compared, based on information about the variable parts of the first image and the second image; an output unit that outputs the mismatched portion detected by the variable portion comparison unit; the variable portion comparison unit changes a method for detecting the mismatch based on information about the variable portion; When the information about the variable portion includes information that the variable portion changes linearly, an image at the start of the change, and an image at the end of the change, The program is characterized in that the variable part comparison unit determines whether the image possessed by the variable part of the second image matches the image in the middle of changing the image at the start of the change to the image at the end of the change.

2. the variable section comparison section calculates a ratio of each of CMYK for each pixel when the CMYK values ​​of the image at the start of the change are changed to the CMYK values ​​of the image at the end of the change; 2. The program according to claim 1, wherein it is determined for each pixel whether there is a timing of change that matches the CMYK ratio of the image contained in the variable portion of the second image.

3. An information processing device a variable part comparison unit that detects mismatches between the variable parts of a first image serving as a reference and a second image to be compared, based on information about the variable parts of the first image and the second image; an output unit that outputs the mismatched portion detected by the variable portion comparison unit; the variable portion comparison unit changes a method for detecting the mismatch based on information about the variable portion; When the information about the variable portion includes information that the variable portion changes based on a pattern and all images that constitute the pattern, The program is characterized in that the variable part comparison unit compares the image contained in the variable part of the second image with all images that make up a pattern, thereby determining whether there is a matching image among all the images.

4. 4. The program according to claim 3, wherein the variable portion comparison unit records the mismatched pixels in association with coordinates for the image with the fewest mismatched pixels among all the images constituting the pattern.

5. An information processing device a variable part comparison unit that detects mismatches between the variable parts of a first image serving as a reference and a second image to be compared, based on information about the variable parts of the first image and the second image; an output unit that outputs the mismatched portion detected by the variable portion comparison unit; the variable portion comparison unit changes a method for detecting the mismatch based on information about the variable portion; When the information about the variable part includes information indicating that the variable part is a date and time and includes images of numbers 0 to 9, The variable part comparison unit compares the image contained in the variable part of the second image with number images of 0 to 9 to determine whether there is a matching number image among the number images of 0 to 9.

6. If there is a matching number image among the number images 0 to 9, 6. The program according to claim 5, wherein the variable portion comparison unit determines whether the matched numeric image is a correct value as a date or time.

7. An information processing device a variable part comparison unit that detects mismatches between the variable parts of a first image serving as a reference and a second image to be compared, based on information about the variable parts of the first image and the second image; an output unit that outputs the mismatched portion detected by the variable portion comparison unit; the variable portion comparison unit changes a method for detecting the mismatch based on information about the variable portion; the variable section comparison section records the fact that there is a mismatch along with the coordinates of the mismatched portion, The information processing device a difference information arrangement recognition unit that detects the arrangement of differences based on continuous mismatches, the ratio of mismatches within a certain area, and a predetermined pattern; a determination unit that determines whether or not there is a difference arrangement that is visible to a human being, based on the difference arrangement recognized by the difference information arrangement recognition unit; A program to function as a

8. 8. The program according to claim 7, wherein the output unit outputs the number of mismatched portions and whether or not there is a difference in the layout that is visible to a human.

9. a variable part comparison unit that detects mismatches between the variable parts of a first image serving as a reference and a second image to be compared, based on information about the variable parts of the first image and the second image; an output unit that outputs the mismatched portion detected by the variable portion comparison unit, the variable portion comparison unit changes a method for detecting the mismatch based on information about the variable portion; When the information about the variable portion includes information that the variable portion changes linearly, an image at the start of the change, and an image at the end of the change, An information processing device characterized in that the variable part comparison unit determines whether the image possessed by the variable part of the second image matches the image in the middle of changing the image at the start of the change to the image at the end of the change.

10. A determination method performed by an information processing device, comprising: a step in which a variable portion comparison unit detects mismatched portions of the variable portions of the first image and the second image based on information about the variable portions of the first image as a reference and the second image to be compared; an output unit outputting the mismatched portion detected by the variable portion comparison unit; the variable portion comparison unit changes a method for detecting the mismatch based on information about the variable portion, When the information about the variable portion includes information that the variable portion changes linearly, an image at the start of the change, and an image at the end of the change, A judgment method characterized in that the variable part comparison unit determines whether the image possessed by the variable part of the second image matches the image in the middle of changing the image at the start of the change to the image at the end of the change.

Images