Image determination method and image determination device

Abstract

To effectively detect blur of a captured image in imaging an object to be read with a recognition target written thereon.SOLUTION: An image determination method includes: an acquisition step for acquiring a captured image obtained by imaging an object with a character to be recognized, the character being written in a predetermined format including a character recognition area in which the character is to be written; an extraction step for extracting a plurality of blur determination areas from the captured image, the blur determination areas being defined in association with the predetermined format and having a predetermined character printed thereon in advance; and a determination step for determining whether or not the captured image is a blurred image by using the predetermined character included in a blur determination area determined from among the plurality of blur determination areas on the basis of a positional relationship with the character recognition area.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present disclosure relates to an image assessment method and an image assessment device. [Background technology]

[0002] Conventionally, in devices such as mobile terminals, images of objects are acquired and information within the images is read. For example, Patent Document 1 discloses a configuration for image recognition in which a predetermined marker is detected from a captured image and information indicating how the object is reflected is acquired based on image information of the marker. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-161174 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure has been devised in consideration of the above-described conventional circumstances, and aims to provide an image assessment method and an image assessment device that more effectively detect blur in a captured image that includes a recognition target. [Means for solving the problem]

[0005] The present disclosure provides an image judgment method including: an acquisition step of acquiring a captured image obtained by photographing an object on which characters to be recognized are written in a predetermined format including a character recognition area in which the characters are written; an extraction step of extracting from the captured image a plurality of blur judgment areas in which predetermined characters are pre-printed and defined in correspondence with the predetermined format; and a judgment step of judging whether or not the captured image is a blurred image using the predetermined characters included in a blur judgment area that is determined from the plurality of blur judgment areas based on a positional relationship with the character recognition area.

[0006] The present disclosure also provides an image assessment device having an acquisition unit that acquires a captured image obtained by photographing an object on which characters to be recognized are written in a predetermined format including a character recognition area in which the characters are written, an extraction unit that extracts from the captured image a plurality of blur assessment areas in which predetermined characters are pre-printed and which are defined in correspondence with the predetermined format, and a assessment unit that determines whether or not the captured image is a blurred image using the predetermined characters included in the blur assessment area that is determined based on the positional relationship between the plurality of blur assessment areas and the character recognition area. Effect of the Invention

[0007] According to the present disclosure, blur in a captured image including a recognition target can be detected more effectively. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an external view for explaining a use case of a terminal device according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a block diagram showing an example of a device configuration of a terminal device according to an embodiment of the present invention; [Diagram 3] FIG. 1 is an explanatory diagram for explaining blur in a captured image; [Figure 4] FIG. 1 is a diagram showing an example of a format of a voucher according to an embodiment of the present invention. [Diagram 5] FIG. 1 is a diagram showing an example of the configuration of a slip according to an embodiment of the present invention; [Figure 6] Conceptual diagram to explain the rolling shutter [Figure 7] FIG. 1 is a diagram for explaining a position of interest corresponding to a scanning direction according to an embodiment of the present invention; [Figure 8] FIG. 1 is a diagram showing an example of a configuration of a UI screen according to an embodiment of the present invention; [Figure 9] 1 is a flowchart of a process according to an embodiment of the present invention; [Figure 10] FIG. 1 is a conceptual diagram for explaining a modified example according to an embodiment of the present invention; [Figure 11]FIG. 1 is a conceptual diagram for explaining a modified example according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] (Background to this disclosure) In conventional character recognition sites, when photographing is performed by manually operating both or either the operating terminal side and the photographing target side, focusing may not be performed correctly on the intended position of the photographing target, resulting in blurring of the photographed image. When character recognition processing is performed on a photographed image containing such blurring, a correct recognition result may not be obtained. Character recognition is also performed on handwritten characters. Handwritten characters vary greatly depending on the person and the situation, and when character recognition is performed on such characters, it is more important to suppress the blurring contained in the photographed image.

[0010] Furthermore, character recognition processing for detecting characters from a captured image generally requires a high processing load, and therefore, in order to prevent unnecessary repetition of character recognition processing, it is necessary to determine, before character recognition is performed, whether the captured image is in an appropriate state for more accurate character recognition, i.e., whether the degree of blur is low.

[0011] Hereinafter, with reference to the drawings as appropriate, each embodiment that specifically discloses the configuration and operation of the image judgment method and image judgment device according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0012] <Embodiment 1> FIG. 1 is a diagram for explaining a use case of a terminal device 100 capable of executing an image assessment method according to the first embodiment of the present invention.

[0013] The use case of the terminal device 100 according to the present embodiment is an example, and is not limited thereto. For example, the terminal device 100 may be a terminal device equipped with a camera 103 capable of photographing an object. In addition, an example of the read object in the present embodiment 1 is a slip 400 for a package to be transported, but is not limited thereto. For example, the read object may be an object having a predetermined format and a predetermined area in which characters are printed, such as a medical questionnaire, application form, order form, etc. In addition, a user of the slip 400 can input characters by handwriting or the like in the predetermined area. In addition, in the present embodiment, the terminal device 100 and the slip 400 are described assuming that one or both of them are held by a user or the like, and hand shaking or the like occurs.

[0014] Terminal device 100 includes a camera 103 and performs image processing on a captured image of slip 400. Furthermore, terminal device 100 detects blur from the captured image (specifically, slip 400 shown in the captured image) and determines whether the captured image is a blurred image (hereinafter referred to as "blur determination"). Based on the result of the blur determination, terminal device 100 performs OCR (Optical Character Recognition / Reader) processing (hereinafter referred to as "character recognition processing") on the captured image to recognize characters written on slip 400. Note that in this embodiment, a configuration will be described in which terminal device 100 performs blur detection and character recognition processing, which will be described later, but terminal device 100 may also be configured to cooperate with an external device and perform part of the processing on the external device side (for example, an external server configured to be able to communicate with terminal device 100).

[0015] [Device configuration] 2 is a block diagram showing an example of the configuration of the terminal device 100 according to the present embodiment. The terminal device 100 according to the present embodiment will be described taking as an example a mobile terminal that is held in the hand of a user and operated, such as a smartphone, a tablet terminal, or a POS terminal. However, the present invention is not limited to this, and the terminal device 100 may be a terminal fixed at a predetermined position. In this case, the user may place an object to which the slip 400 is attached within the angle of view of the camera 103 provided in the terminal device 100, thereby acquiring a captured image including the slip 400 attached to the object.

[0016] The control unit 101 is configured using, for example, a central processing unit (CPU), a graphical processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), etc., and controls the operation of each unit of the terminal device 100. The control unit 101 performs various processes and controls in cooperation with the memory 102. Specifically, the control unit 101 refers to programs and data stored in the memory 102 and executes the programs to realize various functions as described below. As an example, an image processing unit 101A is shown as a part that performs blur determination and character recognition as described below.

[0017] The memory 102 is a volatile / non-volatile storage device, and is configured to be able to hold the program and data according to the present embodiment. The memory 102 may be configured, for example, with a flash memory, a ROM (Read Only Memory), or a RAM (Random Access Memory). The camera 103 is used to capture an image of an object located in the vicinity of the terminal device 100 and obtain a captured image. The camera 103 is configured to have at least a lens (not shown) and an image sensor (not shown). The image sensor is, for example, a solid-state imaging element such as a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and converts an optical image formed on an imaging surface into an electrical signal. The imaging parameters, imaging range, etc. of the camera 103 may be arbitrarily set based on a user operation, etc.

[0018] The input unit 104 is a user interface for accepting various operations on the terminal device 100 by the user. The display unit 105 is a user interface for displaying various information to the user. The display unit 105 may be configured using a display such as an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence). A touch panel display in which the input unit 104 and the display unit 105 are integrated may be used. The communication unit 106 is an interface configured to be able to communicate with an external device (not shown). The communication unit 106 may be capable of performing communication via short-range wireless communication such as Bluetooth (registered trademark) or NFC (Near Field Communication) (registered trademark), or a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark).

[0019] [Photo] FIG. 3 is a diagram for explaining a captured image captured by the camera 103 according to the present embodiment. FIG. 3(a) shows an example of a captured image 300, showing a state without blur (hereinafter referred to as a "normal image"). The captured image 300 includes an area 301 for writing a name and an area 302 for writing a phone number. It is assumed that a character can be detected more accurately by performing character recognition processing on the normal image. FIG. 3(b) shows an example of a captured image 310, showing a state with blur (hereinafter referred to as a "blurred image"). The captured image 310 also includes an area 311 for writing a name and an area 312 for writing a phone number. When character recognition processing is performed on a blurred image, the accuracy of character detection is lower than that of a normal image as shown in FIG. 3(a). Therefore, it is desirable to obtain a captured image in which the blur is more suppressed.

[0020] FIG. 4 is a diagram showing an example of a form of a slip according to the present embodiment. Here, two forms 400 and 410 are given as examples for explanation. The forms 400 and 410 show examples of a form format for a parcel delivery slip. The form includes areas for writing, for example, a postal code, a telephone number, an address, a name, and a delivery date as information about the delivery destination (destination), and characters indicating these. The form also includes areas for writing, for example, a postal code, a telephone number, an address, and a name as information about the requester (requester), and characters indicating these. Characters in the format are printed at predetermined positions on the slip as specified characters. Format information related to the areas and character arrangement in the form of the slip is held in advance in the terminal device 100.

[0021] For example, on the slip 400 shown in Fig. 4(a), the postal code "XXX-YYYY", the telephone number "0123-456-789", the address "XYZ number, C, A city, A prefecture", the name "○○○ △△", the delivery date "1st (Mon) 5th (Sun)", etc. are handwritten as information about the destination to which the delivery service will be delivered. Also, on the slip 400, the postal code "XXX-YXYY", the telephone number "1111-222-333", the address "Room ZZZ, F, D city, A prefecture", the name "XXX □□", etc. are handwritten as information about the requester (i.e., the customer) who will use the transportation service.

[0022] The font, edge strength, color, arrangement, etc. of the characters printed on the slip are predefined. On the other hand, characters written by hand, etc., vary greatly in pen type, character size, character type, handwriting, bleeding, etc. Therefore, it is easier to distinguish whether or not a printed character is blurred than characters written by hand, etc. Therefore, in this embodiment, blur determination is performed using characters preprinted on the slip. When a slip that holds format information is photographed by camera 103, terminal device 100 identifies each area in the photographed image and uses the characters preprinted in the identified areas for blur determination. One type of format information for a slip may be used in one terminal device 100, or multiple types may be used.

[0023] FIG. 5 is a conceptual diagram showing the difference in areas corresponding to the purpose of blur determination according to this embodiment in the format of slip 400 shown in FIG. 4(a). As described above, slip 400 includes a plurality of areas. Of these, area 401 is an area where the user writes by hand or the like. On the other hand, area 402 is an area where characters or the like are printed on slip 400 in advance. Information related to the arrangement of these characters and the printed characters is also included in the format information. Note that in this embodiment, the information used for blur determination is described as "characters", but this includes information that is printed in advance and can be identified, such as logos, icons, and marks.

[0024] [Rolling shutter support] Here, the rolling shutter function considered in this embodiment will be described. The rolling shutter function is known in which, when photographing a slip or the like with the camera 103, one image is acquired by dividing the image into a plurality of areas and scanning the area. Fig. 6 is a diagram for explaining the concept of the rolling shutter function. The scanning direction of the rolling shutter function is the horizontal direction and the vertical direction, and Fig. 6 shows an example of the vertical direction.

[0025] Assume here that the shooting range 600 is divided into three regions 600a, 600b, and 600c. The number of divisions is not particularly limited. Then, these regions are scanned vertically from the top to bottom to obtain one captured image. At this time, the acquisition timing of each region is different. For example, the region 600a is acquired at time T1, the region 600b is acquired at time T2, and the region 600c is acquired at time T3.

[0026] Assuming such a function, the state of camera shake, etc. may differ at each state of time T1, T2, and T3. In other words, the degree of blur may differ depending on the region within one captured image. Therefore, in this embodiment, the region for determining blur is switched within one captured image in consideration of the directionality of scanning by the rolling shutter function, etc.

[0027] Fig. 7 is a diagram for explaining the correspondence between the format information of the slip 410 when performing blur determination according to the present embodiment and the scanning direction in the rolling shutter function. Fig. 7(a) shows an example of the relationship when corresponding to the horizontal rolling shutter function. Fig. 7(b) shows an example of the relationship when corresponding to the vertical rolling shutter function.

[0028] In the case of Fig. 7(a), it is assumed that multiple areas are scanned from left to right (horizontally). The slip 410 includes areas 411a and 411b as areas for handwriting, which are extracted using the captured image and format information corresponding to the slip 410. The slip 410 also includes areas 412a and 412b in which predefined characters are printed, which are also extracted using the captured image and format information corresponding to the slip 410.

[0029] At this time, for example, due to the characteristics of the rolling shutter function, the state (degree) of blur may differ between the area 411a and the area 411b. Therefore, in this embodiment, an area for determining whether blur has occurred in each handwritten area is predefined. For example, in the case of FIG. 7(a), the area 411a determines whether blur has occurred based on the analysis result of the area 412a. Also, the area 411b determines whether blur has occurred based on the analysis result of the area 412b. These correspondences are predefined in the format information in association with the application or non-application of the rolling shutter function and the scanning direction. The area for handwriting is an area for recognizing the written characters, and is hereinafter also referred to as a "recognition area". On the other hand, an area in which characters are printed in advance and the characters are used for blur determination is hereinafter also referred to as a "blur determination area".

[0030] 7(b), it is assumed that multiple areas are scanned from top to bottom (vertically). Slip 410 includes areas 411c to 411j as recognition areas, which are extracted based on the captured image and format information corresponding to slip 410. Furthermore, slip 410 also includes areas 412c and 412j that can be used as blur determination areas, which are also extracted based on the captured image and format information corresponding to slip 410.

[0031] At this time, for example, due to the characteristics of the rolling shutter function, the state (degree) of blur may differ between the regions 411c to 411j. Therefore, in the case of Fig. 7(a), the regions 411c to 411j determine whether or not blur has occurred based on the analysis results of the corresponding regions 412c to 412j. These correspondences are defined in advance in the format information in association with the application or non-application of the rolling shutter function and the scanning direction.

[0032] Here, in the slip format information, each area may be specified to have a different shape or number depending on the scanning direction of the rolling shutter. For example, each area may be set so that the length in the direction along the scanning direction of the rolling shutter is as long as possible and the length in the direction perpendicular to the scanning direction of the rolling shutter is as short as possible. The examples of each area shown in Figures 7(a) and 7(b) are set based on this idea.

[0033] For example, in Fig. 7(a), "zip code" and "address" belong to the same area, but in Fig. 7(b), they belong to different areas. The reason for setting each area in this way is as follows. As explained using Fig. 6, in the rolling shutter function, a single captured image is created by dividing the area in the scanning direction and integrating images captured in sequence. Therefore, the divided captured range is long in the direction perpendicular to the scanning direction (horizontal direction in the example of Fig. 6) and short in the direction along the scanning direction (vertical direction in the example of Fig. 6). Since this range is captured at the same time, it is highly likely that the blur characteristics are similar. Therefore, by setting the shape of each area based on the above-mentioned idea, images with similar blur characteristics can be captured efficiently.

[0034] Each region may be divided more finely in the direction perpendicular to the scanning direction. Although this increases the processing load, it makes it possible to perform more detailed blur determination, such as when the upper half of the paper is not blurred but the lower half is blurred because the paper is curved.

[0035] In this embodiment, the scanning direction of the rolling shutter function has been described as being vertical or horizontal on the paper surface of FIG. 7. However, the scanning direction of the rolling shutter function may be diagonal, such as when shooting with a tilted camera. In such a case, the format used for the blur determination may be determined depending on whether the scanning direction of the rolling shutter function is closer to the vertical or horizontal direction. In addition, a format for when the scanning direction is diagonal may be set separately and used.

[0036] The closer the distance between the recognition area and the blur judgment area, the more preferable. This is because it is assumed that the degree of blur between the close areas is the same. Therefore, the recognition area and the blur judgment area are associated so that the distance between them is closer. In other words, when a usable blur judgment area exists around a certain recognition area, the closer blur judgment area is specified or selected so as to be associated. It is also more preferable to determine the positions of the recognition area and the blur judgment area in consideration of the division position of the rolling shutter function. This is because, when the recognition area and the blur judgment area are included in the range photographed at the same timing by the rolling shutter function, it is highly likely that the degree of blur occurring in the recognition area and the blur judgment area is also the same. In addition, the blur judgment area or the recognition area may straddle the division position (boundary) of the rolling shutter function. In this case, the timing at which a part of the blur judgment area or a part of the recognition area is photographed differs from the other parts, but the timing at which the areas are photographed is close between adjacent areas, and it is highly likely that the degree of blur is the same.

[0037] When the recognition area and the blur judgment area are associated with each other, the more pairs there are, the higher the accuracy of the blur judgment will be, but the higher the processing load will be. Therefore, the area for which the blur judgment is performed may be specified according to the processing speed or the more important parts of the slip. For example, the number and association of the blur judgment area and the recognition area may be determined taking into consideration the width and height of the area. Furthermore, the correspondence is not necessarily limited to one-to-one, and one blur judgment area may be associated with multiple recognition areas.

[0038] [UI screen example] Fig. 8 is a diagram showing an example of a UI (User Interface) screen according to the present embodiment. Fig. 8(a) shows an example of a UI screen 800 displayed when character recognition is performed on a captured image. Fig. 8(b) shows an example of a UI screen 810 for confirming re-capturing (re-acquiring an image) when a captured image is blurred.

[0039] 8A includes a message 801, an image 802, a recognition result 803, an OK button 805, and an NG button 806. The message 801 includes the result of the blur determination process according to this embodiment and a message related to the result. For example, if the captured image is determined to be a blurred image as a result of the blur determination process and character recognition is performed on the captured image, a message such as "The captured image is blurred. Please check it carefully" may be included to prompt confirmation.

[0040] Image 802 shows the area of ​​the captured image where character recognition was performed, i.e., the original image of the recognition area on the slip. If the captured image itself is a blurred image, the image displayed here is likely to be blurred as well. Recognition result 803 includes area 804 that displays the results of performing character recognition on image 802. Here, the characters "0801234" are recognized and output from image 802. Note that if one captured image contains multiple recognition areas, multiple images 802 and recognition results 803 may be provided corresponding to them.

[0041] The OK button 805 is a button that is pressed when the user checks the contents of the recognition result 803 and determines that character recognition has been performed correctly. When the OK button 805 is pressed, the UI screen 800 is closed and the operation may proceed to the next operation. The NG button 806 is a button that is pressed when the user checks the contents of the recognition result 803 and determines that character recognition has not been performed correctly. When the NG button 806 is pressed, the UI screen 800 is closed and the captured image is acquired again.

[0042] As a result of the character recognition, the display in area 804 may be switched between characters with high accuracy and characters with low accuracy. For example, characters with lower accuracy, i.e., characters that require confirmation by the user, may be displayed in a color that is more noticeable to the user. Area 804 may be configured to allow the user to correct the result of the character recognition. In this case, the user may correct the result of the character recognition with the correct character via input unit 104.

[0043] 8(b) includes a message 811, a YES button 812, and a NO button 813. The message 811 includes the result of the blur determination process according to this embodiment and a message related thereto. For example, if the captured image is determined to be a blurred image as a result of the blur determination process, a message such as "The captured image is blurred. Would you like to capture again?" may be included to encourage recapture in response to the blurred image.

[0044] The YES button 812 is a button that is pressed when photographing again. When the YES button 812 is pressed, the UI screen 810 is closed and a transition to an operation screen (not shown) for photographing again may be made. The NO button 813 is a button that is pressed when it is determined that photographing again is not necessary. When the NO button 813 is pressed, the UI screen 810 is closed and a process other than photographing may be executed.

[0045] [Processing flow] 9 is a flowchart of the character recognition process according to this embodiment. This process flow may be realized, for example, by the control unit 101 of the terminal device 100 reading and executing a program or data stored in the memory 102, thereby controlling each part of the terminal device 100. This process flow may be started when a user of the terminal device 100 instructs the start of the character recognition process according to this embodiment. Also, here, it is assumed that the camera 103 photographs the slip using a rolling shutter function, and that the scanning direction, the number of divided areas, and the like are preset.

[0046] The terminal device 100 photographs the slip as the subject of the photographing process via the camera 103, and acquires the photographed image (step S901). Note that the photographed image is not limited to an image photographed in real time, and may be configured to acquire an image photographed and stored at a different timing.

[0047] The terminal device 100 identifies the scanning direction of the rolling shutter function for the captured image acquired in step S901 (step S902). At this time, the terminal device 100 may further acquire information such as the number of divisions of the rolling shutter function and other capture settings.

[0048] The terminal device 100 extracts the blur judgment region and the recognition region from the captured image acquired in step S901 according to the scanning direction specified in step S902 (step S903). The extraction here may be performed, for example, based on the format information associated with the slip after identifying the type of slip shown in the captured image. The type of slip may be specified by the user or may be specified from information included in the captured image. For example, a mark, logo, figure, etc. for uniquely identifying the type of slip may be shown on the slip. The method of extracting the region from the captured image is not particularly limited, and a known method may be used. For example, region extraction may be performed using a trained model obtained by machine learning, or region extraction may be performed by template matching. In addition, a plurality of pairs of the blur judgment region and the recognition region may be extracted, and the number of these may be specified in the format information. In addition, in this process, preprocessing may be performed on the captured image. Examples of preprocessing may include keystone correction, reduction processing, filtering, etc. The reduction process may be configured to reduce the captured image acquired in step S901 to a predetermined size. For example, if the original image size is 3000×2000 pixels, the image may be reduced to a smaller size such as 600×400 pixels. This pre-processing can reduce the processing load of the subsequent image processing.

[0049] The terminal device 100 extracts edges of the print information included in the blur determination area acquired in step S903 (step S904). The print information is information of characters printed in advance as described above. The method of extracting edges is not particularly limited, and any known method may be used.

[0050] The terminal device 100 calculates the statistics of the strength of each edge extracted in step S904 (step S905). The statistics here may be, for example, the maximum value, minimum value, average value, or variance value of pixel values ​​(luminance values). A known method may be used to calculate the statistical values ​​of edge strength. Alternatively, in the vicinity of an edge of a character printed in advance, it may be determined whether or not the image is blurred based on the difference in luminance between the character portion and the luminance of a nearby position. At this time, a threshold value corresponding to a blurred image and a normal image may be stored in advance, and the determination may be made by comparing the calculated statistical value with this threshold value. As described above, the type of character printed in advance in the blur determination area is specified, so that edge determination can be performed according to the configuration of the character.

[0051] The terminal device 100 judges whether the captured image is a blurred image based on the edge intensity calculated in step S905 (step S906). The judgment here may be made based on a predefined threshold value. Alternatively, when one captured image includes a plurality of blur judgment regions, the blurred image judgment may be made for the entire captured image based on the results for each of the blur judgment regions. For example, when the blur judgment region judged to be a blurred image among the plurality of blur judgment regions is equal to or greater than a predetermined threshold value, the entire captured image may be judged to be a blurred image. In addition, when the blur judgment process is performed for a plurality of blur judgment regions, the judgment result for each blur judgment region may be weighted according to the scanning direction of the rolling shutter function to perform the blur judgment for the entire image. For example, the judgment result of the blur judgment region closer to the character recognition region may be prioritized according to the scanning direction of the rolling shutter function. Note that, when the image reduction process is performed in step S903, the judgments in steps S904 to S905 may be performed for the image of the original image size. This is because when an image is reduced, edges are lost, and there is a risk that an accurate blur determination cannot be made on the reduced image.

[0052] As a result of the process of step S906, the terminal device 100 determines whether the captured image is a blurred image (step S907). If the captured image is a blurred image (step S907: YES), the process of the terminal device 100 proceeds to step S910. On the other hand, if the captured image is not a blurred image (step S907: NO), the process of the terminal device 100 proceeds to step S908.

[0053] The terminal device 100 executes character recognition processing on the recognition area (step S908). If the captured image has been reduced in step S903, the recognition area is extracted from the reduced image, and character recognition processing is performed on the extracted recognition area. This reduces the load of the character recognition processing. Note that, in this embodiment, the image on which the processing of step S908 is performed is an image that has been determined not to be a blurred image, so that even if character edge information is lost due to the reduction processing, the accuracy of character recognition is unlikely to decrease. If multiple recognition areas are extracted, the terminal device 100 applies character recognition processing to each recognition area. Then, the processing of the terminal device 100 proceeds to step S909.

[0054] The terminal device 100 generates a UI screen (not shown) including the result of the character recognition process in step S908, and outputs it on the display unit 105 (step S909). At this time, the terminal device 100 may also display that the captured image is not a blurred image. Then, the process proceeds to step S914.

[0055] In response to the determination that the image is a blurred image, the terminal device 100 generates a UI screen 810 indicating that fact and notifies the user (step S910). Then, the terminal device 100 accepts an instruction for the blurred image. In the case of the example of the UI screen 810 in Fig. 8(b), if the YES button 812 is pressed, it is determined that an instruction for re-shooting has been accepted, and if the NO button 813 is pressed, it is determined that an instruction for re-shooting has not been accepted.

[0056] In response to the determination that the image is a blurred image in step S907, the terminal device 100 determines whether or not an instruction to re-shoot has been received (step S911). If an instruction to re-shoot has been received (step S911: YES), the process of the terminal device 100 returns to step S901 and repeats the process. In step S901, if the configuration is other than capturing real-time captured images, a UI screen (not shown) that allows the user to select a non-blurred image from among the stored images may be displayed. If there is no instruction to re-shoot (step S911: NO), the process of the terminal device 100 proceeds to step S912. The instruction to re-shoot is not limited to the timing at which the image is determined to be a blurred image, and may be configured to be set in advance. In other words, when a blurred image is detected, it may be set to transition to a shooting screen that automatically prompts the user to re-shoot, assuming that the accuracy of character recognition will decrease. In such a configuration, the process of notifying the user in step S910 may be omitted.

[0057] The terminal device 100 executes character recognition processing on the recognition area (step S912). If the captured image has been reduced in step S903, the recognition area is extracted from the original captured image, and character recognition processing is performed on the extracted recognition area. The image to be subjected to character recognition processing in step S912 is an image that has been determined to be a blurred image but re-capture has been refused. This is because the original captured image is more likely to enable character recognition with higher accuracy than an image in which edge and other information has been lost due to reduction processing. However, if reducing the load of character recognition processing is more important than accuracy, character recognition processing may be performed on the reduced image. In step S912, the user of the terminal device 100 also executes character recognition processing after recognizing that the image has been determined to be a blurred image, and therefore the user may be aware that the recognition result may be inaccurate and may be trying to reduce the work time. If multiple recognition areas are extracted, the terminal device 100 applies character recognition processing to each recognition area. The algorithm of the character recognition process in step S908 and the algorithm of the character recognition process in step S912 may be different or the same. Since the character recognition process in step S912 is performed on a blurred image, the character recognition process may be performed using parameters that assume a blurred image. In this case, the process branch in step S911 depending on whether or not a re-shooting instruction has been issued may be omitted, and step S912 may be automatically performed. Then, the process of the terminal device 100 proceeds to step S913.

[0058] The terminal device 100 generates a UI screen 800 including the result of the character recognition process in step S912, and outputs it on the display unit 105 (step S913). At this time, the terminal device 100 also displays a message 801 indicating that the captured image is a blurred image and prompting the user to confirm it, as shown in FIG. 8(a).

[0059] The terminal device 100 determines whether or not a correction instruction has been received via the UI screen 800 (step S914). In the case of the UI screen 800 of FIG. 8(a), this determination may be made based on whether or not the user has made an input in the area 804 and pressed the OK button 805. If a correction instruction has been received (step S914: YES), the processing of the terminal device 100 proceeds to step S915. On the other hand, if a correction instruction has not been received (step S914: NO), the processing of the terminal device 100 proceeds to step S916.

[0060] The terminal device 100 updates the character recognition result based on the correction instruction received from the user, and stores the updated result in the memory 102 (step S915). Then, this processing flow ends.

[0061] The terminal device 100 stores the result of the character recognition in the memory 102 (step S916), and then ends this processing flow.

[0062] The character information recognized in this processing flow may be used in other processing in the terminal device 100, or may be transmitted to an external device and used on the external device side.

[0063] As described above, the image determination method according to the present embodiment obtains a captured image obtained by photographing an object (e.g., slips 400, 410) on which characters to be recognized are written in a predetermined format including a character recognition area in which the characters are written, extracts from the captured image a plurality of blur determination areas (e.g., area 402) on which predetermined characters are pre-printed and defined in association with the predetermined format, and determines whether the captured image is a blurred image or not using the predetermined characters included in the blur determination area determined from the plurality of blur determination areas based on the positional relationship with the character recognition area. With this configuration, blur in a captured image including a recognition target can be detected more effectively. In particular, it becomes possible to more accurately determine whether the image is a blurred image by using pre-printed character information.

[0064] In addition, the captured image is judged to be a blurred image or not based on the judgment results of each of the plurality of blur judgment areas corresponding to the position of the character recognition area. With this configuration, it is possible to perform a blur judgment of the entire captured image based on the judgment results of the plurality of blur judgment areas included in the format. Therefore, it is possible to judge the blurred image with higher accuracy.

[0065] In addition, a rolling shutter function is used when capturing an image, and the position of the blur determination area in the predetermined format differs depending on the scanning direction of the rolling shutter function. With this configuration, it is possible to execute a blur determination process corresponding to the rolling shutter function.

[0066] In addition, when a blur determination area close to the character recognition area and a blur determination area far from the character recognition area are extracted in the scanning direction of the rolling shutter function, the result of the determination as to whether or not the captured image is a blurred image based on the blur determination area close to the character recognition area is prioritized over the result of the determination as to whether or not the captured image is a blurred image based on the blur determination area far from the character recognition area. With this configuration, blur determination is performed on the blur determination area located closer to the character recognition area, which is assumed to have the same degree of blur as the character recognition area, making it possible to realize a more accurate blur determination.

[0067] In addition, when a blur judgment area close to the character recognition area and a blur judgment area far from the character recognition area are extracted in the scanning direction of the rolling shutter function, it is determined whether the captured image is a blurred image or not based on the blur judgment area closest to the character recognition area. With this configuration, blur judgment is performed on the blur judgment area located closer to the character recognition area, which is assumed to have the same degree of blur as the character recognition area, making it possible to realize a more accurate blur judgment.

[0068] Furthermore, the shape of the blur determination area in the predetermined format differs depending on the scanning direction of the rolling shutter function. With this configuration, it is possible to execute blur determination processing compatible with the rolling shutter function.

[0069] In addition, when it is determined that the captured image is a blurred image, the captured image is acquired by photographing the object again. With this configuration, when it is determined that the image is a blurred image, the image is automatically acquired again, which makes it possible to improve the convenience for the user.

[0070] In addition, if the captured image is determined to be a blurred image, a screen is displayed that prompts the user to re-capture the object. With this configuration, if the captured image is determined to be a blurred image, the user is prompted to capture the image again, which improves user convenience.

[0071] In addition, when character recognition processing is performed on the character recognition area and the captured image is determined to be a blurred image, the result of the character recognition processing and a message indicating that the captured image is a blurred image are output. With this configuration, character recognition processing can be applied to a captured image determined to be a blurred image, and further, it becomes possible to prompt confirmation of the recognition result.

[0072] [Variation 1] In the above embodiment, a configuration has been described in which one captured image is obtained in one shooting operation, and blur determination and character recognition processing are performed on this captured image. A first modified example will now be described.

[0073] FIG. 10 is a conceptual diagram for explaining the first modified example of the present embodiment. The horizontal axis indicates the passage of time t. Depending on the function of the camera 103, it is possible to obtain multiple captured images in one capturing operation. For example, a video consisting of multiple images (frames) may be obtained, and multiple captured images may be obtained by extracting images from the video at a predetermined interval. Therefore, in the first modified example, the terminal device 100 performs multiple capturing operations in response to one capturing instruction from the user (step S1001). In the example of FIG. 10, it is assumed that four capturing operations, steps S1001a to S1001d, are performed in response to one capturing operation, and four captured images are obtained. At this time, since the capturing timing of each captured image is different, the degree of blur may also differ.

[0074] Then, the terminal device 100 executes a blur determination process for each of the multiple captured images acquired in step S1001 (step S1002). That is, the blur determination process is executed four times in parallel for the four captured images (steps S1002a to S1002d). The blur determination here is equivalent to the contents of steps S902 to S906 in FIG. 9.

[0075] Next, the terminal device 100 specifies the captured image with the least degree of blur as a result of the determination process in steps S1002a to S1002d (step S1003). At this time, if there is a captured image determined to be a normal image, the terminal device 100 selects that captured image.

[0076] Then, the terminal device 100 executes character recognition processing on the captured image selected in step S1003 (step S1004), which reduces the number of operations by the user and enables more accurate character recognition.

[0077] According to the above-mentioned modification 1, multiple captured images are acquired simultaneously in parallel in response to one user operation, and extraction of blur determination regions and blur image determination are performed simultaneously in parallel for the multiple captured images. With this configuration, it becomes possible to acquire multiple captured images and perform blur determination with one user operation.

[0078] [Variation 2] In the above-described first modification, a configuration has been described in which a plurality of captured images are simultaneously acquired in parallel in one shooting operation, and blur determination and character recognition are performed on the captured images.

[0079] 11 is a conceptual diagram for explaining a second modified example of the present embodiment. The horizontal axis indicates the passage of time t. In the second modified example, the terminal device 100 sequentially repeats image capture and blur determination in response to one capture instruction from the user, and performs character recognition when a normal image is acquired. In other words, multiple captured images are continuously acquired.

[0080] After receiving a shooting instruction from a user, the terminal device 100 performs a first shooting operation (step S1101a). Then, the terminal device 100 performs a blur determination on the captured image acquired in step S1101a (step S1102a). While the blur determination is being performed on the captured image captured in the first shooting operation, the terminal device 100 performs a second shooting operation (step S1101b). Since the image shooting timings are different, the degree of blur of the captured images acquired in the sequential shooting operations is different.

[0081] Then, when the terminal device 100 determines that the captured image is a normal image in the blur determination, it ends the shooting operation and the blur determination at that point. After that, the terminal device 100 executes character recognition processing on the captured image determined to be a normal image (step S1103). This reduces the number of operations by the user and enables more accurate character recognition.

[0082] The number of images captured simultaneously in the first modification and the upper limit for repeating the image capturing operation in the second modification may be arbitrarily set by the user. The adjustment may be made in consideration of the processing load, responsiveness, and the like.

[0083] According to the above-mentioned modified example 2, in response to one user operation, photographed images are continuously acquired, extraction of a blur determination area and blur determination are performed for each photographed image acquired, and acquisition of the photographed image is terminated when the photographed image is determined to be not a blurred image. With this configuration, it becomes possible to acquire multiple photographed images and perform blur determination with one user operation.

[0084] [Other variations] In the above embodiment, when the terminal device 100 determines that the captured image is a blurred image, it displays a message prompting the user to retake the image. However, when it is determined that the captured image is a blurred image, it may perform a process for correcting or suppressing the blur, such as edge emphasis. In this case, a message may be displayed to prompt the user to select whether or not to correct the blur, or the blur may be automatically corrected without the user's confirmation. In addition, since the process of correcting the blur can eliminate minor blur but it is difficult to eliminate large blur, the terminal device 100 may switch between correcting the blur or displaying a message depending on the determined degree of blur. In addition, when the terminal device 100 determines that the captured image is a blurred image, it may control so that the algorithm or model used for character recognition is changed to one for blurred images. For example, when it is determined that the captured image is a blurred image, it is possible to use a model that has been machine-learned on only blurred images. Such a model may have a lower accuracy when the image is not a blurred image (a normal image), so it is preferable to use it only when the captured image is determined to be a blurred image.

[0085] In the above embodiment, the correspondence between each recognition area and the blur judgment area is predefined in the format information. However, the type and range of each area may be predefined in the format information, and the blur judgment for each recognition area may be determined at the time of shooting. In this case, the blur judgment area closest to each recognition area in the scanning direction of the rolling shutter function is set as the blur judgment area corresponding to the recognition area. In this way, it is highly likely that the recognition areas and the blur judgment areas photographed almost simultaneously will be associated with each other. However, if it is not necessary to perform the blur judgment strictly, the blur judgment area that is second or later closest to each recognition area may be set as the blur judgment area corresponding to the recognition area. In this case, the order of distance between the recognition area and the blur judgment area may be predefined.

[0086] Depending on the shooting angle, there may be cases where the blur judgment area does not exist in the scanning direction of the rolling shutter function as viewed from the recognition area. In such a case, the closest blur judgment area to each recognition area may be set as the blur judgment area corresponding to that recognition area without considering the scanning direction of the rolling shutter function. If it is not necessary to perform the blur judgment strictly, the second or subsequent closest blur judgment area may be set as the blur judgment area corresponding to that recognition area. Similarly, even if the camera employs a global shutter method in which all pixels are photographed at the same time, the closest blur judgment area to each recognition area or the second or subsequent closest blur judgment area may be set as the blur judgment area corresponding to that recognition area. In other words, the blur judgment area associated with each recognition area may be determined based on the positional relationship between the recognition area and the blur judgment area.

[0087] In the above embodiment, characters are used to determine blur. This is because characters generally include straight lines in various directions, making it easy to determine blur at various angles, and because, in the case of slips and the like, there is a high possibility that characters indicating the contents to be filled are printed. However, other information, such as the frame lines of the slip, may also be used to determine blur.

[0088] In the above embodiment, the character recognition result or message is described as being output on a screen, but it may be output in other forms. For example, the recognition result or message may be output as a voice, or the recognition result or message may be output to a recording medium of the terminal device 100 or an external server, etc., without notifying the user.

[0089] In the above embodiment, the determination of whether the captured image is blurred may be performed for each recognition area. As described above, the blur determination area is set for each recognition area based on the distance from each recognition area, etc. Therefore, the determination result of the blur determination area corresponding to each recognition area may be treated as the blur determination result for that recognition area. As a result, if it is determined that some recognition areas are not blurred and other recognition areas are blurred, only the recognition areas determined to be not blurred may be subject to character recognition, and the recognition results may be output. In addition, various methods are conceivable for handling the recognition areas determined to be blurred. For example, the result of character recognition may not be output, or a message indicating that character recognition is performed but that there is a high possibility that errors are included may be output.

[0090] According to the above modification, the predetermined format includes a plurality of character recognition areas, and it is determined whether or not the captured image is a blurred image based on the blur determination area selected for each of the plurality of character recognition areas, and if the captured image is determined to be not a blurred image based on the blur determination area selected for some of the plurality of character recognition areas, and the captured image is determined to be a blurred image based on the blur determination area selected for the other character recognition areas, character recognition processing is performed on the part of the character recognition areas, and the result of the character recognition processing of the part of the character recognition areas is output. With this configuration, even if a part of the captured image includes blur, character recognition processing can be applied to the character recognition areas determined to be not blurred, making it possible to improve the convenience of the user regarding character recognition.

[0091] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the various embodiments described above may be arbitrarily combined within the scope of the invention.

[0092] (Additional Note) The above-described embodiments disclose the following techniques. (Technology 1) an acquisition step of acquiring a captured image by photographing an object on which characters to be recognized are written in a predetermined format including a character recognition area in which the characters are to be written; an extraction step of extracting from the captured image a plurality of blur determination regions in which predetermined characters are printed in advance and which are defined in association with the predetermined format; a determination step of determining whether or not the captured image is a blurred image by using the predetermined character included in a blur determination area determined based on a positional relationship between the plurality of blur determination areas and the character recognition area; The image evaluation method includes the steps of: With this configuration, it is possible to more effectively detect blur in a captured image including a recognition target. In particular, it is possible to more accurately determine whether an image is blurred by using character information printed in advance.

[0093] (Technology 2) The image judgment method according to Technology 1, in which in the judgment step, it is judged whether the captured image is a blurred image or not based on the judgment results of each of the plurality of blur judgment areas corresponding to the positions of the character recognition areas. This configuration makes it possible to perform blur determination for the entire captured image based on the determination results for a plurality of blur determination regions included in the format, thereby enabling more accurate determination of blurred images.

[0094] (Technology 3) A rolling shutter function is used when capturing the captured image, The image assessment method according to Technology 1 or Technology 2, wherein the position of a blur assessment area in the predetermined format differs depending on a scanning direction of the rolling shutter function. This configuration makes it possible to execute a blur determination process compatible with the rolling shutter function.

[0095] (Technology 4) In the image judgment method according to Technology 3, when a blur judgment area close to the character recognition area and a blur judgment area far from the character recognition area are extracted in the scanning direction of the rolling shutter function, in the judgment step, a judgment result of whether or not the captured image is a blurred image based on a blur judgment area close to the character recognition area is given priority over a judgment result of whether or not the captured image is a blurred image based on a blur judgment area far from the character recognition area. With this configuration, blur judgment is performed on a blur judgment area that is closer to the character recognition area and is assumed to have the same degree of blur as the character recognition area, making it possible to achieve more accurate blur judgment.

[0096] (Technology 5) In the image judgment method according to the fourth aspect, when a blur judgment area close to the character recognition area and a blur judgment area far from the character recognition area are extracted in the scanning direction of the rolling shutter function, in the judgment step, it is judged whether or not the captured image is a blurred image based on the blur judgment area closest to the character recognition area. With this configuration, blur judgment is performed on a blur judgment area that is closer to the character recognition area and is assumed to have the same degree of blur as the character recognition area, making it possible to achieve more accurate blur judgment.

[0097] (Technology 6) The image determination method according to any one of Techniques 3 to 5, wherein the shape of the blur determination area in the predetermined format differs depending on the scanning direction of the rolling shutter function. With this configuration, it is possible to execute a blur determination process corresponding to the rolling shutter function.

[0098] (Technology 7) The image judgment method according to any one of Technology 1 to Technology 6, wherein if the captured image is judged to be a blurred image in the judgment process, the captured image is acquired by re-capturing the object in the acquisition process. With this configuration, when it is determined that the image is blurred, the image is automatically acquired again, thereby improving user convenience.

[0099] (Technology 8) The image judgment method according to any one of Technology 1 to Technology 6, further comprising a display step of displaying a screen prompting a user to re-photograph the object when the captured image is judged to be a blurred image in the judgment step. With this configuration, if it is determined that the image is blurred, the user is prompted to capture the image again, making it possible to improve user convenience.

[0100] (Technology 9) a recognition step of performing character recognition processing on the character recognition area, an output step of outputting a result of the character recognition process and a message indicating that the captured image is a blurred image when the captured image is determined to be a blurred image in the determination step; The image assessment method according to any one of techniques 1 to 6, comprising: With this configuration, character recognition processing can be applied to a captured image that has been determined to be a blurred image, and further, it becomes possible to prompt the user to confirm the recognition result.

[0101] (Technology 10) the predetermined format includes a plurality of character recognition regions; In the determination step, it is determined whether or not the captured image is the blurred image based on a blur determination area selected for each of the plurality of character recognition areas; When the captured image is determined to be not the blurred image based on the blur determination area selected for some of the character recognition areas among the plurality of character recognition areas, and the captured image is determined to be the blurred image based on the blur determination area selected for the other character recognition areas, the recognition step performs character recognition processing on the part of the character recognition areas, The output step outputs a result of the character recognition process of the part of the character recognition area. The image assessment method described in Technology 10. With this configuration, even if part of the captured image is blurred, character recognition processing can be applied to character recognition areas that are determined to be not blurred, thereby improving user convenience regarding character recognition.

[0102] (Technology 11) Acquiring a plurality of photographed images simultaneously and in parallel in response to one user operation by the acquiring step; The image judgment method according to any one of Techniques 1 to 6, wherein the extraction step and the judgment step are performed simultaneously in parallel on the plurality of captured images. This configuration makes it possible to obtain a plurality of captured images and perform blur determination with a single operation by the user.

[0103] (Technology 12) The photographed images are continuously acquired by the acquiring step in response to one user operation; performing the extraction step and the determination step each time the photographed image is acquired; The image judgment method according to any one of Techniques 1 to 6, wherein the acquisition step is terminated at a point in time when the captured image is judged not to be a blurred image in the judgment step. This configuration makes it possible to obtain a plurality of captured images and perform blur determination with a single operation by the user.

[0104] (Technology 13) an acquisition unit that acquires a captured image by photographing an object on which characters to be recognized are written in a predetermined format including a character recognition area in which the characters are to be written; an extracting unit that extracts from the captured image a plurality of blur determination regions in which predetermined characters are printed in advance and which are defined in association with the predetermined format; a determination unit that determines whether the captured image is a blurred image by using the predetermined character included in a blur determination area that is determined based on a positional relationship between the plurality of blur determination areas and the character recognition area; An image assessment device having the above structure. With this configuration, it is possible to more effectively detect blur in a captured image including a recognition target. In particular, it is possible to more accurately determine whether an image is blurred by using character information printed in advance. [Industrial Applicability]

[0105] INDUSTRIAL APPLICABILITY The present disclosure is useful as an image assessment method and an image assessment device that more effectively detect blur in a captured image when imaging a reading target on which a recognition target is written. [Explanation of symbols]

[0106] 100...Terminal device 101...Control unit 102…Memory 103...Camera 104...Input section 105...Display section 106…Communications Department

Claims

1. an acquisition step of acquiring a captured image by photographing an object on which characters to be recognized are written in a predetermined format including a character recognition area in which the characters are to be written; an extraction step of extracting from the captured image a plurality of blur determination regions in which predetermined characters are printed in advance and which are defined in association with the predetermined format; a determination step of determining whether or not the captured image is a blurred image by using the predetermined character included in a blur determination area determined based on a positional relationship between the plurality of blur determination areas and the character recognition area; The image evaluation method includes the steps of:

2. The image determination method according to claim 1 , wherein in the determination step, it is determined whether or not the captured image is a blurred image based on a determination result for each of the plurality of blur determination areas corresponding to the position of the character recognition area.

3. A rolling shutter function is used when capturing the captured image, The image assessment method according to claim 1 , wherein a position of a blur assessment region in the predetermined format differs depending on a scanning direction of the rolling shutter function.

4. 4. The image judgment method according to claim 3, wherein, when a blur judgment area close to the character recognition area and a blur judgment area far from the character recognition area are extracted in the scanning direction of the rolling shutter function, in the judgment step, a judgment result of whether or not the captured image is a blurred image based on a blur judgment area close to the character recognition area is given priority over a judgment result of whether or not the captured image is a blurred image based on a blur judgment area far from the character recognition area.

5. 5. The image determination method according to claim 4, wherein, when a blur determination area close to the character recognition area and a blur determination area far from the character recognition area are extracted in the scanning direction of the rolling shutter function, in the determination step, it is determined whether or not the captured image is a blurred image based on the blur determination area closest to the character recognition area.

6. The image assessment method according to claim 3 , wherein a shape of a blur assessment region in the predetermined format differs depending on a scanning direction of the rolling shutter function.

7. The image judging method according to claim 1 , wherein if the captured image is judged to be a blurred image in the judging step, the captured image is acquired by re-capturing an image of the object in the acquiring step.

8. The image judgment method according to claim 1 , further comprising a display step of displaying a screen prompting a user to re-image the object when the captured image is judged to be a blurred image in the judgment step.

9. a recognition step of performing character recognition processing on the character recognition area, an output step of outputting a result of the character recognition process and a message indicating that the captured image is a blurred image when the captured image is determined to be a blurred image in the determination step; The image assessment method according to claim 1 , further comprising:

10. the predetermined format includes a plurality of character recognition regions; In the determination step, it is determined whether or not the captured image is the blurred image based on a blur determination area selected for each of the plurality of character recognition areas; When the captured image is determined to be not the blurred image based on the blur determination area selected for some of the character recognition areas among the plurality of character recognition areas, and the captured image is determined to be the blurred image based on the blur determination area selected for the other character recognition areas, the recognition step performs character recognition processing on the part of the character recognition areas, The output step outputs a result of the character recognition process of the part of the character recognition area. The image assessment method according to claim 9.

11. Acquiring a plurality of photographed images simultaneously and in parallel in response to one user operation by the acquiring step; The image assessment method according to claim 1 , wherein the extraction step and the assessment step are performed simultaneously and in parallel on the plurality of captured images.

12. The photographed images are continuously acquired by the acquiring step in response to one user operation; performing the extraction step and the determination step each time the photographed image is acquired; The image judging method according to claim 1 , wherein the acquiring step is terminated when the captured image is judged not to be a blurred image in the judging step.

13. an acquisition unit that acquires a captured image by photographing an object on which characters to be recognized are written in a predetermined format including a character recognition area in which the characters are to be written; an extracting unit that extracts from the captured image a plurality of blur determination regions in which predetermined characters are printed in advance and which are defined in association with the predetermined format; a determination unit that determines whether the captured image is a blurred image by using the predetermined character included in a blur determination area that is determined based on a positional relationship between the plurality of blur determination areas and the character recognition area; An image assessment device having the above structure.

Images