Screen comparison device, screen comparison method, and program

Abstract

To provide a screen comparison apparatus configured to efficiently and accurately confirm unintentional change of words or arrangement before and after a correction when altering an application screen.SOLUTION: A screen comparison apparatus 100 for comparing screens to be displayed by an application to be corrected includes an extraction unit 111 and a detection unit 112. The extraction unit 111 extracts, as an OCR result of a current screen, at least one character included in the current screen displayed by the application before the correction, and extracts, as an OCR result of a new screen, at least one character included in the new screen displayed by the application after the correction. The detection unit 112 detects an error of the character on the new screen, in accordance with a pair of at least one OCR result of the current screen and at least one OCR result of the new screen extracted by the extraction unit 111.SELECTED DRAWING: Figure 15

Description

[Technical field]

[0001] The present invention relates to a screen comparison device, a screen comparison method, and a program. [Background technology]

[0002] When updating an application, the application is tested by visually comparing the screen before and after the revision to ensure that there are no unintended changes in wording or button layout. For example, a technology for a program that tests Web applications has been disclosed (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-116496 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, because the screen before and after the correction is compared visually, there is a possibility that errors or omissions in a long character string may be detected incorrectly. In addition, when changing the screen size, it is difficult to visually compare the screen before and after the correction, so there is a possibility that an error may occur in the confirmation. In addition, because it is necessary to visually check each screen, the confirmation work takes time.

[0005] In view of the above-mentioned problems, the present invention aims to provide a screen comparison device that can efficiently and accurately check for unintended changes in wording and layout before and after modification when updating an application screen. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the present invention provides a screen comparison device for comparing screens displayed by an application to be corrected, the device comprising: an extraction unit that extracts at least one or more characters included in a current screen displayed by the application before correction as a character recognition result for the current screen, and extracts at least one or more characters included in a new screen displayed by the application after correction as a character recognition result for the new screen, and a detection unit that detects errors in characters in the new screen according to pairs of at least one or more character recognition results for the current screen and at least one or more character recognition results for the new screen extracted by the extraction unit.

[0007] The present invention also provides a screen comparison method executed by a screen comparison device which compares screens displayed by an application to be corrected, comprising the steps of: extracting at least one or more characters contained in a current screen displayed by the application before correction as a character recognition result for the current screen; extracting at least one or more characters contained in a new screen displayed by the application after correction as a character recognition result for the new screen; and detecting character errors in the new screen according to pairs of the extracted at least one or more character recognition results for the current screen and the at least one or more character recognition results for the new screen.

[0008] The present invention is also characterized in that it is a program executed by a screen comparison device that compares screens displayed by an application to be corrected, extracting at least one or more characters included in a current screen displayed by the application before correction as a character recognition result for the current screen, and extracting at least one or more characters included in a new screen displayed by the application after correction as a character recognition result for the new screen, and executing a process to detect errors in characters in the new screen according to pairs of the extracted at least one or more character recognition results for the current screen and at least one or more character recognition results for the new screen. Effect of the Invention

[0009] According to the present invention, when updating an application screen, it is possible to efficiently and accurately check for unintended changes in wording and layout before and after modification. [Brief description of the drawings]

[0010] [Figure 1] 1A and 1B are diagrams showing examples of screens to be compared by the screen comparison device of the present embodiment; [Diagram 2] FIG. 2 is a hardware configuration diagram of the screen comparison device according to the present embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of functional blocks of the screen comparison device relating to the screen comparison process of the present embodiment. [Figure 4] 10 is a flowchart showing an example of a procedure of a screen comparison method by a screen comparison process executed by the screen comparison device. [Diagram 5] 11 is a flowchart illustrating an example of a procedure of an extraction process executed by the screen comparison device. [Figure 6] 11A and 11B are diagrams illustrating an example of an OCR result array of a current screen and an OCR result array of a new screen stored in a storage unit. [Figure 7] 13 is a flowchart illustrating an example of a procedure for a single pair extraction process. [Figure 8] 13 is a flowchart illustrating an example of a procedure for a multiple pair extraction process. [Figure 9] 11 is a diagram showing an example of an outline of a method for acquiring an OCR result of a current screen acquired by a surrounding OCR result acquisition process. FIG. [Figure 10] 11 is a diagram showing an example of an outline of a method for acquiring an OCR result of a new screen acquired by a surrounding OCR result acquisition process. FIG. [Figure 11] 13 is a flowchart illustrating an example of a procedure for a surrounding OCR result acquisition process. [Figure 12] 13 is a flowchart illustrating an example of a procedure for an OCR result update process. [Figure 13] 13 is a flowchart showing an example of a procedure for a typographical error and omission check process. [Figure 14]10 is a flowchart illustrating an example of a procedure for a position deviation check process. [Figure 15] FIG. 13 is a diagram showing an example of notifying typos and misalignment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] FIG. 1 is a diagram showing an example of screens compared by a screen comparison device 100 of this embodiment. The screen comparison device 100 is used, for example, as a device for comparing screens displayed by an application to be modified. Hereinafter, a screen displayed by an application before modification is referred to as a current screen 10, and a screen displayed by an application after modification is referred to as a new screen 20. As an application, the screen comparison device 100 can be applied to an application of an ATM (automatic teller machine) such as an automated teller machine. Applications to which the screen comparison device 100 is applied are not limited to ATMs, and the screen comparison device 100 can be applied to various applications in which it is essential to display a screen.

[0012] 1, new screen 20 displayed by the application after correction displays wording 300a that was not displayed on current screen 10 displayed by the application before correction, resulting in a typographical error. Furthermore, on current screen 10 displayed by the application before correction, wording 300b was displayed in the bottom center of the screen, but on new screen 20 displayed by the application after correction, wording 300b is displayed on the bottom left side of the screen, resulting in a typographical error. Furthermore, the spelling of wording 300c displayed on current screen 10 displayed by the application before correction, "English Guide," is displayed differently from the spelling of wording 300c displayed on new screen 20 displayed by the application after correction, "English Guid," resulting in a typographical error.

[0013] Although the current screen 10 and new screen 20 should display the same text, typos and omissions may occur as shown in Figure 1. When comparing typos and omissions visually, it is cumbersome to check every time a new screen 20 is created. Furthermore, relying solely on human visual inspection can lead to errors in the check, as it is difficult to compare the screens before and after correction. Furthermore, since it is necessary to visually check each screen, the check process takes time.

[0014] In this embodiment, a screen comparison device 100 is described that can efficiently and accurately check for unintended wording or changes in layout before and after modification when updating an application screen.

[0015] The screen comparison device 100 has the functions of a computer such as a workstation, a server, a personal computer, etc., and executes a screen comparison process based on a screen comparison method, which will be described later. The screen comparison device 100 may be communicably connected to other devices such as a server through a network (not shown).

[0016] Fig. 2 is a hardware configuration diagram of the screen comparison device 100 in this embodiment. As shown in Fig. 2, the screen comparison device 100 is configured by connecting a control unit 101, an input device 102, an output device 103, a storage unit 104, and an interface (I / F) 105 to a bus 106. Furthermore, the screen comparison device 100 may include other hardware configurations not shown in Fig. 2.

[0017] The control unit 101 comprehensively controls the entire screen comparison device 100, and includes a CPU (Central Processing Unit) that reads programs and executes screen comparison processing.

[0018] The input device 102 is, for example, a keyboard, a joystick, a light pen, a mouse, a touch pad, a touch panel, a trackball, or the like, and inputs various types of data and signals.

[0019] The output device 103 is, for example, various displays such as an LCD (Liquid Crystal Display), a printer, etc., and outputs images and other information.

[0020] The storage unit 104 has a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), etc. The storage unit 104 stores a program for the screen comparison device 100 to execute the screen comparison process, as well as a control program for controlling and executing each function of the screen comparison device 100, and screen data such as a current screen and a new screen. The storage unit 104 provides a working area when the screen comparison device 100 executes a program. The storage unit 104 may also store a current screen OCR result array and a new screen OCR result array (described later) that store the results of comparing the character recognition result of the current screen with the character recognition result of the new screen.

[0021] The interface 105 is a unit for connecting to an external device (not shown), such as a serial interface such as a Universal Serial Bus (USB) or a parallel interface such as Ethernet (registered trademark). The current screen 10 and the new screen 20 to be compared by the screen comparison device 100 are stored in the storage unit 104 via the interface 105.

[0022] The screen comparison process in this embodiment is executed by the screen comparison device 100 described above.

[0023] Fig. 3 is a diagram showing an example of functional blocks of the screen comparison device 100 relating to the screen comparison process of this embodiment. As shown in Fig. 3, the control unit 101 of the screen comparison device 100 includes an extraction unit 111, a detection unit 112, and a notification unit 113. The control unit 101 may include other components not shown in Fig. 3. The storage unit 104 stores a current screen OCR result array 121 and a new screen OCR result array 122, which will be described later. The storage unit 104 may store information not shown in Fig. 3.

[0024] The extraction unit 111 extracts at least one or more words included in the current screen 10 displayed by the application before modification as a character recognition result of the current screen 10. The extraction unit 111 uses an existing optical character recognition (OCR) technology to extract at least one or more words from the words included in the current screen 10 as a character recognition result of the current screen 10 (hereinafter referred to as an "OCR result").

[0025] Furthermore, the extraction unit 111 extracts at least one or more words contained in the new screen 20 displayed by the modified application as a character recognition result of the new screen 20. The extraction unit 111 uses an existing optical character recognition technology to extract at least one or more words from the words contained in the new screen 20 as a character recognition result of the new screen 20 (hereinafter referred to as an "OCR result").

[0026] The detection unit 112 detects character errors in the new screen 20 according to pairs of at least one or more OCR results of the current screen 10 extracted by the extraction unit 111 and at least one or more OCR results of the new screen 20 .

[0027] The notification unit 113 notifies the user of the character error on the new screen 20 detected by the detection unit 112 through the display of the output device 103. The notification unit 113 may notify the user by sound through a speaker (not shown).

[0028] Next, a specific description will be given of the screen comparison processing executed by the screen comparison device 100. Fig. 4 is a flowchart showing an example of the procedure of a screen comparison method by the screen comparison processing executed by the screen comparison device 100. The screen extraction processing executed by the screen comparison method is executed by the control unit 101 of the screen comparison device 100.

[0029] First, the control unit 101 performs an extraction process (step S101). In the extraction process, a process is performed to extract the OCR result included in the current screen 10 and to extract the OCR result included in the new screen 20. The extraction process executed by the screen comparison device 100 will be specifically described with reference to FIG. 5.

[0030] Fig. 5 is a flowchart showing an example of the procedure of the extraction process executed by the screen comparison device 100 in step S101 of Fig. 4. The extraction process executed by the screen comparison device 100 is executed by the control unit 101 of the screen comparison device 100.

[0031] First, the control unit 101 performs OCR on the current screen 10 and extracts the OCR result (step S201). In this process, the control unit 101 extracts at least one pre-correction character included in the current screen 10 displayed by the pre-correction application as the OCR result of the current screen 10.

[0032] The control unit 101 stores the OCR results of the current screen extracted in step S201 in the current screen OCR result array 121 (step S202). As a result of the processes of steps S201 and S202, the control unit 101 determines whether or not all the OCR results of the current screen have been extracted (step S203). If all the OCR results of the current screen have not been extracted (step S203: NO), the processes of steps S201 to S203 are repeatedly executed until all the OCR results of the current screen are extracted.

[0033] If all OCR results for the current screen have been extracted (step S203: YES), the control unit 101 performs OCR on the new screen and extracts the OCR results (step S204). In this process, the control unit 101 extracts at least one corrected phrase contained in the new screen displayed by the corrected application as the OCR result for the new screen. The OCR result includes characters displayed on the new screen 20.

[0034] The control unit 101 stores the OCR results of the new screen extracted in step S204 in the OCR result array of the new screen (step S205). As a result of the processing of steps S204 and S205, the control unit 101 judges whether or not all the OCR results of the new screen have been extracted (step S206). If all the OCR results of the new screen have not been extracted (step S206: NO), the processing of steps S204 to S206 is repeatedly executed until all the OCR results of the new screen have been extracted. If all the OCR results of the new screen have been extracted (step S206: YES), the extraction processing executed by the screen comparison device 100 is terminated. When this processing is terminated, the processing proceeds to step S102 in FIG. 4.

[0035] FIG. 6 is a diagram showing an example of the current screen OCR result array 121 and the new screen OCR result array 122 stored in the storage unit 104. As shown in FIG.

[0036] The current screen OCR result array 121 stores, for each ID, the OCR result of the current screen, the coordinates of the OCR result, a counter of the OCR result of the current screen, and information on the ID of the associated new screen. The OCR result of the current screen stores the text of the result of OCR of the current screen 10. The coordinates of the OCR result store position information where the OCR result of the current screen is arranged.

[0037] The counter for the OCR result of the current screen stores information on the number of times the OCR result is displayed on the current screen 10. The ID of the linked new screen stores the ID of the OCR result of the new screen 20 that corresponds to the OCR result of the current screen 10.

[0038] The new screen OCR result array 122 stores, for each ID, the OCR result of the new screen, the coordinates of the OCR result, a counter of the OCR result of the new screen, the ID of the associated current screen, the check result, and positional deviation information. The new screen OCR result stores the text of the result of OCR of the new screen 20. The coordinates of the OCR result store the position information where the OCR result of the new screen is arranged.

[0039] The counter for the OCR result of the new screen stores information on the number of times the OCR result is displayed on the new screen 20. The ID of the linked new screen stores the ID of the OCR result of the new screen 20 that corresponds to the OCR result of the current screen 10.

[0040] In step S102 of Fig. 4, the control unit 101 performs a single pair extraction process. The single pair extraction process executed in step S102 will be described in detail with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the procedure of the single pair extraction process executed in step S102 of Fig. 4. The single pair extraction process executed by the screen comparison device 100 is executed by the control unit 101 of the screen comparison device 100.

[0041] First, the control unit 101 selects one OCR result of the current screen stored in the current screen OCR result array 121 (step S301).

[0042] The control unit 101 searches the array of OCR results for a new screen that matches at least a part of the OCR result for the current screen selected in step S301 (step S302).

[0043] As a result of the search in step S302, it is determined whether the OCR result of the current screen selected in step S301 matches at least a part of the OCR result of the new screen (step S303). If the OCR result of the selected current screen does not match the OCR result of the new screen (step S303: NO), the process proceeds to step S306.

[0044] If the selected OCR result of the current screen matches at least a part of the OCR result of the new screen (step S303: YES), the control unit 101 increments the counter of the current screen OCR result array 121 in which the matching OCR result is stored (step S304). For example, the counter of an OCR result displayed only once on the current screen 10 becomes "1", and the counter of an OCR result displayed twice on the current screen 10 becomes "2". The control unit 101 links the ID of the OCR result of the new screen corresponding to the matching OCR result to the current screen OCR result array 121 (step S305). For example, the OCR result "Deposit" corresponding to ID "1" of the current screen OCR result array corresponds to the OCR result "Deposit" of ID "2" and "Deposit" of ID "7" of the new screen OCR result array. Therefore, "2" and "7" are stored in the ID of the new screen associated with the OCR result corresponding to ID "1" in the current screen OCR result array.

[0045] Control unit 101 determines whether or not all OCR results stored in current screen OCR result array 121 have been selected (step S306). If all OCR results stored in current screen OCR result array 121 have not been selected (step S306: NO), the process returns to step S301, and steps S301 to S306 are repeatedly executed until all OCR results stored in current screen OCR result array 121 have been selected. If all OCR results stored in current screen OCR result array 121 have been selected (step S306: YES), the process proceeds to step S307.

[0046] The control unit 101 selects one of the OCR results for the new screen stored in the OCR result array for the new screen (step S307).

[0047] The control unit 101 searches the current screen OCR result array 121 for an OCR result of the current screen that matches at least a part of the OCR result of the new screen selected in step S307 (step S308). As a result of the search in step S308, it is determined whether the OCR result of the new screen selected in step S307 matches at least a part of the OCR result of the current screen (step S309).

[0048] If the OCR result of the selected new screen does not match the OCR result of the current screen (step S309: NO), the process proceeds to step S312. If the OCR result of the selected new screen matches at least a part of the OCR result of the current screen (step S309: YES), the control unit 101 increments the counter of the OCR result array of the new screen in which the matching OCR result is stored (step S310). For example, the counter of an OCR result displayed only once on the new screen 20 is "1", and the counter of an OCR result displayed twice on the new screen 20 is "2".

[0049] The control unit 101 links the ID of the OCR result of the current screen corresponding to the matching OCR result to the OCR result array of the new screen (step S311). For example, the OCR result "Deposit" corresponding to ID "2" in the new screen OCR result array corresponds to the OCR result "Deposit" of ID "1" and "Deposit" of ID "6" in the current screen OCR result array. Therefore, "1" and "6" are stored in the ID of the current screen linked to the OCR result corresponding to ID "2" in the new screen OCR result array.

[0050] The control unit 101 determines whether or not all the OCR results stored in the OCR result array of the new screen have been selected (step S312). If all the OCR results stored in the OCR result array of the new screen have not been selected (step S312: NO), the process returns to step S307, and steps S307 to S312 are repeatedly executed until all the OCR results stored in the OCR result array of the new screen have been selected. Then, if all the OCR results stored in the OCR result array of the new screen have been selected (step S312: YES), the single pair extraction process executed by the screen comparison device 100 is terminated. When this process is terminated, the process proceeds to step S103 in FIG. 4.

[0051] In step S103 of Fig. 4, the control unit 101 performs a multiple pair extraction process. The multiple pair extraction process executed in step S103 will be described in detail with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the procedure of the multiple pair extraction process executed in step S103 of Fig. 4. The multiple pair extraction process executed by the screen comparison device 100 is executed by the control unit 101 of the screen comparison device 100.

[0052] First, the control unit 101 selects one OCR result of the current screen stored in the current screen OCR result array 121 (step S401).

[0053] The control unit 101 judges whether the counter of the OCR result selected from the OCR results of the current screen stored in the current screen OCR result array 121 is 2 or more (step S402). In this process, the OCR results of the current screen stored in the current screen OCR result array are processed one item at a time. Then, the process is performed for those with a counter of 2 or more, that is, those with two or more candidates for a pair of OCR results. For example, according to the current screen OCR result array 121 in FIG. 6, the OCR result "Deposit" of ID "1" and the OCR result "Deposit" of ID "6" are to be processed. If the counter of the OCR result of the current screen stored in the current screen OCR result array 121 is 1 (step S402: NO), the process proceeds to step S414.

[0054] If the counter of the OCR result of the current screen stored in the current screen OCR result array 121 is 2 or more (step S402: YES), the control unit 101 selects one OCR result stored in the current screen OCR result array 121 (step S403). The control unit 101 specifies the ID of the OCR result of the current screen selected in step S403 as a parameter (step S404).

[0055] The control unit 101 performs surrounding OCR result acquisition processing (step S405), which will be described later with reference to Fig. 9. In this processing, processing is performed to acquire surrounding OCR results for the OCR results of the current screen stored in the current screen OCR result array 121.

[0056] Fig. 9 is a diagram showing an example of an outline of a method for acquiring the OCR results of the current screen acquired by the surrounding OCR result acquisition process in step S405. As shown in Fig. 9, when "Deposit" is selected from the OCR results of the current screen stored in the current screen OCR result array 121, the control unit 101 acquires the OCR result "Withdrawal" to the right of "Deposit" and the OCR result "Enter in bankbook" below as the surrounding OCR results. In this case, the OCR result "None" above, the OCR result "Enter in bankbook" (ID "2") below, the OCR result "None" to the left, and the OCR result "Withdrawal" (ID "3") to the right are acquired as output data to be stored in the current screen OCR result array 121.

[0057] The control unit 101 selects one of the OCR results of the new screen stored in the OCR result array of the new screen (step S406). The control unit 101 specifies the ID of the OCR result of the new screen selected in step S406 as a parameter (step S407).

[0058] The control unit 101 performs surrounding OCR result acquisition processing, which will be described later with reference to Fig. 9 (step S408). The surrounding OCR result acquisition processing executed in step S408 is substantially the same as the surrounding OCR result acquisition processing executed in step S405, but in the processing executed in step S408, processing is performed to acquire OCR results around the OCR results of the new screen stored in the new screen OCR result array 122.

[0059] Fig. 10 is a diagram showing an example of an outline of a method for acquiring the OCR results of a new screen acquired by the surrounding OCR result acquisition process in step S408. As shown in Fig. 10, when "Deposit" is selected from the OCR results of a new screen stored in the new screen OCR result array 122, the control unit 101 acquires the OCR result "Withdrawal" to the right of "Deposit" and the OCR result "Enter in bankbook" below as the surrounding OCR results. In this case, the OCR result "New screen" (ID "1") above, the OCR result "Enter in bankbook" (ID "2") below, the OCR result "None" to the left, and the OCR result "Withdrawal" (ID "4") to the right are acquired as output data to be stored in the current screen OCR result array 121.

[0060] The control unit 101 determines whether the surrounding OCR results match (step S409). If the surrounding OCR results match (step S409: YES), the control unit 101 sets the counter corresponding to the OCR result in the current screen OCR result array 121 to 1 (step S410). That is, in this case, it is determined that a one-to-one OCR result pair that matches the OCR result has been established. The control unit 101 updates the associated screen ID (step S411). When this process ends, the process proceeds to step S414.

[0061] In the process of step S409, if the surrounding OCR results do not match (step S409: NO), the control unit 101 judges whether the process has been executed the number of times indicated by the counter (step S412). For example, in the case of an OCR result with a counter of "2", it is judged whether the process of steps S406 to S412 has been executed twice. For example, in the case of the OCR result "deposit" with ID "1" in the current screen OCR result array, since the counter is "2", the process is executed twice for the OCR result "deposit" with ID [2] in the new screen OCR result array 122 and the OCR result "deposit" with ID "7". If the process has not been executed the number of times indicated by the counter, the process returns to step S406, and the process of steps S406 to S412 is executed repeatedly until the process has been executed the number of times indicated by all the counters.

[0062] When the process has been repeated for all the counters (step S412), the counter of the selected OCR result in the current screen OCR result array 121 is set to 0 (step S413). That is, in this case, it is determined that there is no OCR result item that matches the OCR result.

[0063] In step S414, the control unit 101 determines whether or not all the OCR results stored in the current screen OCR result array 121 have been selected (step S414). If all the OCR results stored in the current screen OCR result array 121 have not been selected (step S414: NO), the process returns to step S402, and steps S402 to S414 are repeatedly executed until all the OCR results stored in the current screen OCR result array 121 have been selected. Then, if all the OCR results stored in the current screen OCR result array 121 have been selected (step S414: YES), the multiple pair extraction process executed by the screen comparison device 100 is terminated.

[0064] The surrounding OCR result acquisition process executed in step S405 or step S408 in Fig. 8 will be described in detail with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the procedure of the surrounding OCR result acquisition process executed in step S405 or step S408 in Fig. 8. The surrounding OCR result acquisition process executed by the screen comparison device 100 is executed by the control unit 101 of the screen comparison device 100.

[0065] First, the control unit 101 selects one OCR result stored in the current screen OCR result array 121 (step S501). The control unit 101 determines whether the counter of the selected OCR result is 1 or not (step S502). In this process, the control unit 101 excludes an OCR result with a counter of "0" from comparison because it is an OCR result for which no one-to-one pair exists, and an OCR result with a counter of "2" or more is an OCR result for which one-to-one pair candidates cannot be narrowed down. If the counter of the selected OCR result is not 1 (step S502: NO), that is, if the counter of the selected OCR result is 0 or 2 or more, the process proceeds to step S507.

[0066] If the counter for the selected OCR result is 1 (step S502: YES), the control unit 101 selects one of the directions, up, down, left, and right (step S503). For example, the control unit 101 selects the up direction from the up, down, left, and right directions. The control unit 101 determines whether other OCR results exist in the selected direction (step S504). In this process, the control unit 101 uses the coordinate information stored in the current screen OCR result array 121 and the new screen OCR result array 122 to determine in which direction, up, down, left, and right, other OCR results exist. The control unit 101 determines whether it is up, down, left, or right based on the coordinates (Xs, Ys, Xe, Ye) stored in the current screen OCR result array 121 and the coordinates (Xs, Ys, Xe, Ye) stored in the new screen OCR result array 122.

[0067] If another OCR result exists in the selected direction (step S504: YES), the control unit 101 performs an OCR result update process (step S505) which will be described later with reference to Fig. 12. When this process ends, the process proceeds to step S507.

[0068] If there are no other OCR results in the selected direction (step S504: NO), the control unit 101 judges whether or not all directions have been selected (step S506). In this process, the control unit 101 judges whether the processes of steps S503 to S506 have been performed for all directions, including the upward, downward, rightward, and leftward directions. If not all directions have been selected (step S506: NO), the process returns to step S503, and the processes of steps S503 to S506 are repeatedly performed until all directions, including the upward, downward, leftward, and rightward directions, have been selected.

[0069] When all the directions, up, down, left and right, have been selected (step S506: YES), the control unit 101 judges whether or not all the OCR results stored in the current screen OCR result array 121 have been selected (step S507). When all the OCR results stored in the current screen OCR result array 121 have not been selected (step S507: NO), the process returns to step S501, and the processes of steps S501 to S507 are repeatedly executed until all the OCR results stored in the current screen OCR result array 121 have been selected. When all the OCR results stored in the current screen OCR result array 121 have been selected (step S507: YES), the surrounding text acquisition process executed by the screen comparison device 100 is terminated. In the case of the surrounding OCR result acquisition process of step S405 in FIG. 8, the process proceeds to step S406 in FIG. 8. In the case of the surrounding OCR result acquisition process of step S408 in FIG. 8, the process proceeds to step S409 in FIG. 8.

[0070] The OCR result update process executed in step S505 of Fig. 11 will be described in detail with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the procedure of the OCR result update process executed in step S505 of Fig. 11. The OCR result update process executed by the screen comparison device 100 is executed by the control unit 101 of the screen comparison device 100.

[0071] First, the control unit 101 calculates the distance between the OCR result of the current screen selected in the process of step S501 in Fig. 11 and the OCR result of the direction determined to exist in the process of step S504 in Fig. 11 (step S601). In this process, the control unit 101 calculates the distance between the OCR results using the coordinate information stored in the current screen OCR result array 121 and the new screen OCR result array 122.

[0072] Based on the result of calculation in step S601, it is determined whether the distance between the OCR result of the selected current screen and the OCR result in the direction determined to exist is closer than a predetermined distance (step S602). The predetermined distance can be set arbitrarily. For example, a threshold value that determines the predetermined distance can be set arbitrarily depending on the characteristics of the screen, such as the current screen and the new screen displayed by the application. Note that if multiple OCR results exist in the same direction, the closest OCR result is used.

[0073] If the distance between the OCR result of the selected current screen and the OCR result in the direction determined to exist is greater than a predetermined distance (step S602: NO), the OCR result update process performed by the screen comparison device 100 is terminated, and the process proceeds to step S507 in Figure 11.

[0074] If the distance between the OCR result of the selected current screen and the OCR result of the direction determined to exist is closer than a predetermined distance (step S602: YES), the control unit 101 determines whether the OCR result of the direction determined to exist has already been set (step S603).

[0075] If the OCR result for the direction determined to exist has not been set (step S603: NO), the control unit 101 sets the OCR result for the current screen for the determined direction (step S604). When this process ends, the OCR result update process executed by the screen comparison device 100 ends, and the process proceeds to step S507 in FIG.

[0076] If the OCR result for the direction determined to exist has already been set (step S603: YES), the control unit 101 determines whether the distance is closer than the OCR result for the set direction (step S605). If the distance is farther than the OCR result for the set direction (step S605: NO), the OCR result update process executed by the screen comparison device 100 ends, and the process proceeds to step S507 in FIG.

[0077] If the distance is shorter than the OCR result in the set direction (step S605: YES), the control unit 101 updates the OCR result of the current screen in the determined direction (step S606). When this process ends, the OCR result update process executed by the screen comparison device 100 ends, and the process proceeds to step S507 in FIG.

[0078] In step S104 of Fig. 4, the control unit 101 performs a typographical error and omission check process. The typographical error and omission check process executed in step S104 will be described in detail with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the procedure of the typographical error and omission check process executed in step S104 of Fig. 4. The typographical error and omission check process executed by the screen comparison device 100 is executed by the control unit 101 of the screen comparison device 100.

[0079] First, the control unit 101 selects one OCR result stored in the current screen OCR result array 121 (step S701). The control unit 101 compares the OCR result stored in the current screen OCR result array 121, which is linked by an ID, with the OCR result stored in the new screen OCR result array 122 (step S702). In this process, it is determined whether or not at least a part of the OCR result stored in the current screen OCR result array 121 matches with the OCR result stored in the new screen OCR result array 122.

[0080] The control unit 101 adds the comparison result obtained in step S702 to the check results of the new screen OCR result array 122 (step S703).

[0081] For example, when comparing the OCR result "Deposit" of ID "1" stored in the current screen OCR result array 121 with the OCR result "Deposit" of ID "2" stored in the new screen OCR result array 122, the wording is a perfect match. In this case, the control unit 101 adds information indicating "perfect match" to the check result of the new screen OCR result array 122.

[0082] On the other hand, when the OCR result "English Guide" for ID "7" stored in current screen OCR result array 121 is compared with the OCR result "English Guid" for ID "8" stored in new screen OCR result array 122, the wording only matches at the beginning. In this case, control unit 101 adds information indicating "match at the beginning" to the check result of new screen OCR result array 122.

[0083] The control unit 101 determines whether or not all the OCR results stored in the current screen OCR result array 121 have been selected (step S704). If all the OCR results stored in the current screen OCR result array 121 have not been selected (step S704: NO), the process returns to step S701, and the processes of steps S701 to S704 are repeatedly executed until all the OCR results stored in the current screen OCR result array 121 have been selected. If all the OCR results stored in the current screen OCR result array 121 have been selected (step S704: YES), the typo check process executed by the screen comparison device 100 is completed, and the process proceeds to step S105 in FIG. 4.

[0084] In step S105 of Fig. 4, the control unit 101 performs a misalignment check process. The misalignment check process executed in step S105 will be described in detail with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the procedure of the misalignment check process executed in step S105 of Fig. 4. The misalignment check process executed by the screen comparison device 100 is executed by the control unit 101 of the screen comparison device 100.

[0085] First, the control unit 101 selects one OCR result of the current screen stored in the current screen OCR result array 121 (step S801). The control unit 101 specifies the ID of the OCR result of the current screen selected in step S801 as a parameter (step S802). The control unit 101 performs the surrounding OCR result acquisition process described with reference to FIG. 11 (step S803).

[0086] The control unit 101 determines whether or not all the OCR results stored in the current screen OCR result array 121 have been selected (step S804). If all the OCR results stored in the current screen OCR result array 121 have not been selected (step S804: NO), the process returns to step S801, and the processes of steps S801 to S804 are repeatedly executed until all the OCR results stored in the current screen OCR result array 121 have been selected.

[0087] If all OCR results stored in the current screen OCR result array 121 have been selected (step S804: YES), the control unit 101 selects one new screen OCR result stored in the new screen OCR result array 122 (step S805). The control unit 101 specifies the ID of the OCR result of the new screen selected in step S805 as a parameter (step S806). The control unit 101 performs the surrounding OCR result acquisition process described with reference to FIG. 11 (step S807).

[0088] The control unit 101 determines whether or not all the OCR results stored in the new screen OCR result array 122 have been selected (step S808). If all the OCR results stored in the new screen OCR result array 122 have not been selected (step S808: NO), the process returns to step S805, and the processes of steps S805 to S8084 are repeatedly executed until all the OCR results stored in the new screen OCR result array 122 have been selected.

[0089] If all OCR results stored in the new screen OCR result array 122 have been selected (step S808: YES), the control unit 101 compares the surrounding OCR results between the OCR results stored in the current screen OCR result array 121 and the OCR results stored in the new screen OCR result array 122, and stores the comparison results in the new screen OCR result array 122 (step S809).

[0090] For example, a comparison will be made between the OCR result "deposit" of ID "1" stored in the current screen OCR result array 121 and the OCR result "deposit" of ID "2" stored in the new screen OCR result array 122. In this case, the OCR result of the OCR result "deposit" of ID "1" stored in the current screen OCR result array 121 in the upward direction is "none", the OCR result of ...

[0091] Also, a comparison between the OCR result "English Guide" of ID "7" stored in the current screen OCR result array 121 and the OCR result "English Guid" of ID "8" stored in the new screen OCR result array 122 will be described. In this case, the OCR result of the OCR result "English Guide" of ID "7" stored in the current screen OCR result array 121 in the upward direction is "Public utility bill payment", the OCR result of the downward direction is "None", the OCR result of the OCR result of the left direction is "Credit card simple insurance / life insurance", and the OCR result of the OCR result of the right direction is "None". On the other hand, the OCR result of the OCR result "English Guid" of ID "8" stored in the new screen OCR result array 122 in the upward direction is "Public utility bill payment", the OCR result of ... When the process ends, the position shift check process executed by the screen comparison device 100 ends, and the process proceeds to step S106 in FIG.

[0092] In step S106 in Fig. 4, the control unit 101 performs a result notification process. In the result notification process, a process is performed to notify the user of typos, omissions, and misalignment based on the check results and the comparison results stored in the new screen OCR result array 122. Fig. 15 is a diagram showing an example of notifying the user of typos, omissions, and misalignment.

[0093] For example, for an OCR result with a check result of "forward match," highlighting 401 is performed to notify the OCR results of typos and omissions by referencing the coordinate information stored in the current screen OCR result array 121 and the coordinate information stored in the new screen OCR result array 122. In highlighting 401, for example, highlighting is performed using a red frame. This makes it easy to check whether there are unintended typos and omissions in the text before and after correction when updating the application screen. Therefore, checking for typos and omissions in the text can be performed efficiently and accurately.

[0094] Furthermore, for OCR results where the comparison result is "misalignment exists," highlighting 402, 403 is performed to notify the user of the misalignment by referencing the coordinate information stored in the current screen OCR result array 121 and the coordinate information stored in the new screen OCR result array 122. The highlighting 402, 403 is performed by, for example, highlighting with a yellow frame. This makes it easy to check whether there is an unintended change in the layout before and after the correction when updating the application screen. This allows the user to check the layout change efficiently and accurately.

[0095] Although not shown in the figure, OCR results that are free of typos, omissions, and misalignment may be highlighted, for example, with a blue border. Also, highlighting is currently done with red, yellow, and blue borders, but is not limited to this and may be done with blinking, shading, fluorescent colors, etc.

[0096] The present invention is not limited to the above-described embodiment as it is, and the components can be modified and embodied within the scope of the gist of the invention at the implementation stage. In addition, various inventions can be formed by appropriate combinations of multiple components disclosed in the above embodiments. For example, all components shown in the embodiments may be appropriately combined. Furthermore, components across different embodiments may be appropriately combined. Of course, various modifications and applications can be made within the scope of the invention without departing from the spirit of the invention. [Explanation of symbols]

[0097] 10: Current screen 20: New screen 100: Screen comparison device 101: Control unit 102: Input device 103: Output device 104: Storage section 105: Interface 106: Bus 111:Extraction part 112: Detection unit 113: Notification Department 121: Current screen OCR result array 122: New screen OCR result array 300a :Text 300b :Text 300c: Wording 401 : Highlight 402 : Highlight 403 : Highlight

Claims

1. In a screen comparison device for comparing screens displayed by an application to be modified, an extraction unit that extracts at least one character included in a current screen displayed by the application before the modification as a character recognition result for the current screen, and extracts at least one character included in a new screen displayed by the application after the modification as a character recognition result for the new screen; a detection unit that detects an error in a character of the new screen according to a pair of at least one or more character recognition results of the current screen and at least one or more character recognition results of the new screen extracted by the extraction unit; a means for checking the presence or absence of positional deviation of the character recognition results of the at least one or more new screens by comparing the character recognition results of the at least one or more current screens in the upward, downward, rightward, and leftward directions with the character recognition results of the at least one or more new screens in the upward, downward, rightward, and leftward directions, respectively; Equipped with No positional deviation indicates a comparison result in which the character recognition results of the at least one or more current screens in the upward, downward, right, and left directions match the character recognition results of the at least one or more new screens in the upward, downward, right, and left directions, respectively; A screen comparison device characterized in that a position shift indicates a comparison result in which the character recognition result of the at least one or more current screens in a specified direction among the upward, downward, right, and left directions does not match the character recognition result of the at least one or more new screens in the specified direction among the upward, downward, right, and left directions.

2. 2. The screen comparison device according to claim 1, The screen comparison device further comprises a notification unit that notifies a user of an error in characters on the new screen detected by the detection unit.

3. 3. The screen comparison device according to claim 1, The detection unit detects character errors in the new screen based on character recognition results for the upward, downward, right, and left directions of the at least one or more current screens and character recognition results for the upward, downward, right, and left directions of the at least one or more new screens.

4. A screen comparison method executed by a screen comparison device for comparing screens displayed by an application to be modified, comprising: extracting at least one or more characters included in a current screen displayed by the application before the modification as a character recognition result for the current screen, and extracting at least one or more characters included in a new screen displayed by the application after the modification as a character recognition result for the new screen; Detecting an error in a character of the new screen according to the extracted pairs of at least one character recognition result of the current screen and at least one character recognition result of the new screen; checking whether or not there is a positional deviation of the character recognition results of the at least one or more new screens by comparing the character recognition results of the at least one or more current screens in the upward, downward, rightward, and leftward directions with the character recognition results of the at least one or more new screens in the upward, downward, rightward, and leftward directions, respectively; No positional deviation indicates a comparison result in which the character recognition results of the at least one or more current screens in the upward, downward, right, and left directions match the character recognition results of the at least one or more new screens in the upward, downward, right, and left directions, respectively; The screen comparison method is characterized in that a position shift indicates a comparison result in which the character recognition result of the at least one or more current screens in a predetermined direction among the upward, downward, right, and left directions does not match the character recognition result of the at least one or more new screens in the predetermined direction among the upward, downward, right, and left directions.

5. A program executed by a screen comparison device for comparing screens displayed by an application to be corrected, extracting at least one or more characters included in a current screen displayed by the application before the modification as a character recognition result for the current screen, and extracting at least one or more characters included in a new screen displayed by the application after the modification as a character recognition result for the new screen; Detecting an error in a character of the new screen according to the extracted pairs of at least one character recognition result of the current screen and at least one character recognition result of the new screen; checking whether or not there is a positional deviation in the character recognition results of the at least one or more new screens by comparing the character recognition results of the at least one or more current screens in the upward, downward, rightward, and leftward directions with the character recognition results of the at least one or more new screens in the upward, downward, rightward, and leftward directions, respectively; Execute the process, No positional deviation indicates a comparison result in which the character recognition results of the at least one or more current screens in the upward, downward, right, and left directions match the character recognition results of the at least one or more new screens in the upward, downward, right, and left directions, respectively; A program characterized in that a position shift indicates a comparison result in which the character recognition result of the at least one or more current screens in a specified direction among the upward, downward, right, and left directions does not match the character recognition result of the at least one or more new screens in the specified direction among the upward, downward, right, and left directions.

Images