Character recognition device, character recognition method and character recognition program

Abstract

To provide a character recognition device capable of accurately reading a character string engraved on an object without using a rotary stand or the like.SOLUTION: An image acquisition part 21 images an object by a camera 41 for acquiring multiple pieces of image data of the object. A reading part 23 reads the character in each digit from the image data of the object, for each digit of a correct digit number being the digit number of the character string engraved on the object. An estimation part 24 estimates the character in the digit to be estimated, from the character in the digit to be estimated which is read from each of the multiple pieces of image data, the respective digits of the correct digit number being the digits to be estimated.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a technique for reading a character string such as an ID displayed on an object such as a wafer. ID is an abbreviation for IDentifier. [Background technology]

[0002] When processing objects such as wafers, visual inspection is required to ensure that the object is correct. However, errors are likely to occur during visual inspection due to factors such as small characters on the object or characters being difficult to see due to reflections from lighting. As a result, there is a risk of processing the wrong object. In addition, inspection requires time and effort, such as the need to assign separate workers to perform visual inspection.

[0003] Patent Document 1 describes a method in which a wafer is placed on a turntable, and the position where the ID is written is specified based on the position of the notch in the wafer. In Patent Document 1, the wafer is rotated by the turntable to move the position where the ID is written into the shooting area of ​​the camera, and the ID is read. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-261501 A Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, the position where the ID is written is identified, and the position where the ID is written is moved to the shooting area of ​​the camera using a rotating table, thereby enabling the ID to be read with high accuracy. However, in Patent Document 1, it is necessary to place the wafer on the rotating table, which requires a lot of rework if the wafer is incorrect. Also, in Patent Document 1, the wafer is rotated by the rotating table, so the wafer is in contact with the rotating table, which is a piece of equipment, and there is a possibility that the wafer will be scratched. Also, if a scratch is found on the wafer after it has been placed on the rotating table, it is not possible to determine whether the scratch was made before or after the wafer was placed on the rotating table, making it difficult to determine who is responsible. An object of the present disclosure is to enable accurate reading of a character string displayed on an object without using a rotating table or the like. [Means for solving the problem]

[0006] The character recognition device according to the present disclosure comprises: an image acquisition unit that captures an image of an object using a camera and acquires a plurality of image data of the object; a reading unit that reads a character of each digit of a correct answer number, which is the number of digits of a character string shown on the object, from the target image data, using each of the plurality of image data acquired by the image acquisition unit as target image data; an estimation unit that estimates a character of a target digit from the character of the target digit read from each of the plurality of image data by the reading unit, with each digit of the correct digit number being a target digit; Equipped with. Effect of the Invention

[0007] In the present disclosure, characters are read for each digit of the correct number of digits from multiple image data, and characters are estimated for each digit. This makes it possible to read the character string with high accuracy even if the position where the character string is indicated cannot be accurately identified using a rotating table or the like. [Brief description of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of a character recognition device 10 according to a first embodiment. [Diagram 2]3 is a flowchart of the overall process of the character recognition device 10 according to the first embodiment. [Diagram 3] 3 is a diagram showing an example of a correct answer digit number 32, a correct answer character string 33, and a specified digit 34 according to the first embodiment. FIG. [Figure 4] 2 is an explanatory diagram of a method of using the character recognition device 10 according to the first embodiment. [Diagram 5] 5 is an explanatory diagram of a screen display process and a recognition area display process according to the first embodiment. [Figure 6] FIG. 4 is an explanatory diagram of an estimated image display process according to the first embodiment. [Figure 7] FIG. 4 is an explanatory diagram of an estimated character display process according to the first embodiment. [Figure 8] 4 is a flowchart of a reading process according to the first embodiment. [Figure 9] FIG. 4 is an explanatory diagram of an overlap removal process according to the first embodiment. [Figure 10] 4 is a flowchart of a defect filling process according to the first embodiment. [Figure 11] FIG. 4 is an explanatory diagram of a defect setting process according to the first embodiment. [Figure 12] FIG. 4 is an explanatory diagram of a defect filling process according to the first embodiment. [Figure 13] FIG. 4 is an explanatory diagram of a margin calculation process according to the first embodiment. [Figure 14] FIG. 4 is an explanatory diagram of a left side setting process according to the first embodiment. [Figure 15] FIG. 4 is an explanatory diagram of a right-side setting process according to the first embodiment. [Figure 16] FIG. 4 is an explanatory diagram of an estimation process and an inspection process according to the first embodiment. [Figure 17] FIG. 4 is an explanatory diagram of the character confirmation process according to the first embodiment. [Figure 18] FIG. 11 is an explanatory diagram of a guide according to the first modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Embodiment 1 ***Configuration Description*** The configuration of a character recognition device 10 according to the first embodiment will be described with reference to FIG. The character recognition device 10 is a computer. The character recognition device 10 includes the following hardware components: a processor 11, a memory 12, a storage 13, and a communication interface 14. The processor 11 is connected to other hardware components via signal lines and controls these other hardware components.

[0010] The processor 11 is an IC that performs processing. IC is an abbreviation for Integrated Circuit. Specific examples of the processor 11 include a CPU, a DSP, and a GPU. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0011] The memory 12 is a storage device that temporarily stores data. Specific examples of the memory 12 include SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory.

[0012] The storage 13 is a storage device that stores data. A specific example of the storage 13 is an HDD. HDD is an abbreviation for Hard Disk Drive. The storage 13 may also be a portable recording medium such as an SD (registered trademark) memory card, CompactFlash (registered trademark), NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. SD is an abbreviation for Secure Digital. DVD is an abbreviation for Digital Versatile Disk.

[0013] The communication interface 14 is an interface for communicating with an external device. Specific examples of the communication interface 14 include Ethernet (registered trademark), USB, and HDMI (registered trademark) ports. USB is an abbreviation for Universal Serial Bus. HDMI is an abbreviation for High-Definition Multimedia Interface.

[0014] Character recognition device 10 is connected to camera 41 and display device 42 via communication interface 14. Camera 41 is an imaging device for capturing an image of an object. Display device 42 is a display such as an LCD for displaying the reading results and the like. LCD is an abbreviation for Liquid Crystal Display.

[0015] Character recognition device 10 includes, as functional components, an image acquisition unit 21, a reading control unit 22, a reading unit 23, an estimation unit 24, an inspection unit 25, and a display unit 26. The functions of the functional components of character recognition device 10 are realized by software. Storage 13 stores programs that realize the functions of each functional component of character recognition device 10. These programs are loaded into memory 12 by processor 11 and executed by processor 11. In this way, the functions of each functional component of character recognition device 10 are realized.

[0016] The storage 13 stores a background image 31, a correct answer digit number 32, a plurality of correct answer character strings 33, and designated digits 34.

[0017] 1 shows only one processor 11. However, there may be a plurality of processors 11, and the plurality of processors 11 may cooperate to execute programs that realize the respective functions.

[0018] ***Explanation of Operation*** The operation of the character recognition device 10 according to the first embodiment will be described with reference to FIGS. An operation procedure of character recognition device 10 according to embodiment 1 corresponds to a character recognition method according to embodiment 1. Moreover, a program that realizes the operation of character recognition device 10 according to embodiment 1 corresponds to a character recognition program according to embodiment 1.

[0019] In the first embodiment, it is assumed that a set of objects, such as several tens of objects, is to be processed according to an ID. Here, it is assumed that each object is assigned a different ID with the same number of digits.

[0020] The overall process of the character recognition device 10 according to the first embodiment will be described with reference to FIG. As a prerequisite for the process of FIG. 2, a background image 31, a correct answer digit number 32, a plurality of correct answer character strings 33, and a specified digit number 34 are set in the storage 13 by the user. The background image 31 is image data obtained by photographing the camera 41 when there is no object such as a wafer in the photographing area of ​​the camera 41. The correct answer number 32 is the number of digits of the character string shown on the object. In the first embodiment, the correct answer number 32 is the number of digits of the ID shown on the object. The correct answer character string 33 is the character string shown on the object. In the first embodiment, the correct answer character string 33 is the ID shown on the object, and each of the IDs shown on all the objects is set as the correct answer character string 33. The designated digits 34 are one or more digits that must be estimated. In the first embodiment, the designated digits 34 are one or more digits that are required to uniquely identify the ID. For example, it is assumed that the ID assigned to a group of objects is seven digits, and the upper two digits are common. In this case, the lower five digits of the seven digits are designated as the designated digits 34. The correct number of digits 32, the correct character string 33, and the designated digits 34 may be set as one piece of data, for example, as shown in Fig. 3. In Fig. 3, for each digit of the number of digits indicated by the correct number of digits 32, the correct character and whether or not it is a required digit are indicated by the designated digits 34. The upper two digits (the first and second digits) are common parts of the ID, so the required decision is False (not required), and the lower five digits are required digits for identifying the ID, so they are defined as True (required).

[0021] In the first embodiment, as shown in Fig. 4, a worker manually moves an object into the shooting area of ​​camera 41, and reads an ID, which is a character string shown on the object, from image data of the object obtained by camera 41. The worker holds the object while wearing protective gloves or the like to prevent the object from being scratched. If the read ID is the intended ID, the object is ready to be processed.

[0022] (Step S101: Background reading process) The reading control unit 22 reads the background image 31 from the storage 13 .

[0023] (Step S102: First image acquisition process) The image acquisition unit 21 acquires image data obtained by photographing an imaging area with the camera 41. It is assumed that the resolution and the like are specified for the camera 41 in advance.

[0024] (Step S103: Image display process) 5A, the display unit 26 superimposes a guide indicating a recognition area for recognizing characters on the image data acquired in step S102 and displays it on the display device 42. It is assumed that the (x, y) coordinates of the upper left and the (x, y) coordinates of the lower right are specified in advance for the guide.

[0025] (Step S104: Recognition area display process) As shown in FIG. 5B, the display unit 26 displays on the display device 42 an image of the recognition area portion indicated by the guide, out of the image data acquired in step S102.

[0026] (Step S105: Background determination process) The reading control unit 22 compares the image data acquired in step S102 with the background image 31 read in step S101. The reading control unit 22 determines whether the difference between the image data and the background image 31 is equal to or greater than a first threshold value. When the difference is equal to or greater than the first threshold, the reading control unit 22 determines that an object has entered the shooting area of ​​the camera 41, and proceeds to step S106. On the other hand, when the difference is less than the first threshold, the reading control unit 22 determines that an object has not entered the shooting area of ​​the camera 41, and returns the process to step S102. When the process is returned to step S102, new image data is acquired in step S102.

[0027] (Step S106: Previous image setting process) The reading control unit 22 sets the most recently acquired image data as the background image for reading start determination. The most recently acquired image data is the image data acquired in step S102 or the image data acquired in step S107 described later.

[0028] (Step S107: Second image acquisition process) The image acquisition unit 21 acquires image data obtained by photographing an imaging area with the camera 41 .

[0029] (Step S108: Previous image determination process) The reading control unit 22 compares the image data acquired in step S107 with the background image for reading start determination set in step S106. The reading control unit 22 determines whether the similarity of the features between the image data and the background image for reading start determination is less than a second threshold value. If the feature similarity is less than the second threshold, the reading control unit 22 determines that the position of the ID, which is a character string, is being aligned with the recognition area, and returns the process to step S106. On the other hand, if the feature similarity is equal to or greater than the second threshold, the reading control unit 22 determines that the position of the ID, which is a character string, has been aligned with the recognition area, and advances the process to step S109.

[0030] (Step S109: Reading process) The reading unit 23 takes the most recently acquired image data as the target image data, and for each digit of the 32 correct digits, reads the character of that digit from the target image data. When the process of step S109 is executed for the first time, the most recently acquired image data is the image data acquired in step S107. When the process of step S109 is executed for the second time or later, the most recently acquired image data is the image data acquired in step S114.

[0031] (Step S110: Estimation process) The estimation unit 24 estimates the character of the target digit from the characters of the target digit that have been read in step S109 so far, with each digit of the 32 correct digits being treated as a target digit.

[0032] (Step S111: Estimated image display process) As shown in Fig. 6A, the display unit 26 displays the image data used for reading in step S109. Also, as shown in Fig. 6B, the display unit 26 displays on the display device 42 an image of the recognition area portion indicated by the guide, out of the image data used for reading in step S109.

[0033] (Step S112: Estimated character display process) 7, the display unit 26 displays the characters estimated in step S110. Here, only the characters for digits that have been estimated are displayed, and characters for digits that have not been estimated are displayed as blanks.

[0034] (Step S113: Inspection process) The checking unit 25 checks whether the guess has been made using the correct answer character string 33 and the designated digits 34. If the estimation is complete, the inspection unit 25 ends the process. On the other hand, if the estimation is not complete, the inspection unit 25 returns the process to step S102. In step S113, the checking unit 25 checks whether the estimation of the specified digits 34 in which the required item is defined as True has been completed in the correct answer character string 33. Note that the checking unit 25 may be configured to check the entire number of correct answer digits 32 including the required item as False.

[0035] The reading process (step S109 in FIG. 2) according to the first embodiment will be described with reference to FIG. (Step S201: First extraction process) The reading unit 23 cuts out a partial image of an input size for the object detection model from the target image data, with the center position of the recognition area as a reference. For example, the reading unit 23 cuts out a partial image of an input size with the center position of the recognition area as a center position. The object detection model is a trained model that detects characters using image data as input. The input size of the object detection model, which is the size of the image data to be input, is set.

[0036] (Step S202: First character recognition process) The reading unit 23 inputs the partial image cut out in step S201 into the object detection model. The object detection model detects characters from input image data and outputs the position of the detected characters and the recognition result of the detected characters. Here, a rectangular detection frame surrounding the detected characters is output as the position of the characters. The character recognition result indicates which character is detected and the certainty of the recognition.

[0037] (Step S203: Out-of-range removal process) The reading unit 23 removes characters outside the recognition area from among the characters detected by the object detection model in step S202. Specifically, the reading unit 23 removes characters whose detection frames are outside the recognition area by a reference ratio or more. As a result, only characters within the recognition area remain.

[0038] (Step S204: Duplicate removal process) When the degree of overlap between the regions in which the two characters are detected is higher than a reference rate, the reading unit 23 removes the character with the lower degree of certainty of the two characters. A specific description will be given with reference to FIG. 9. There are cases where the detection frames output from the object detection model overlap. There are cases where the detection frames are slightly shifted and the detection frames of adjacent characters overlap, and cases where one character is detected multiple times and the detection frames overlap. When the detection frames of adjacent characters overlap, the reading unit 23 leaves the detection frames as they are. On the other hand, when one character is detected multiple times, the reading unit 23 leaves only one detection frame and removes the remaining detection frames. That is, for the portion where the detection frame U and the detection frame V in FIG. 9 overlap, the detection frame remains as it is. On the other hand, for the portion where the detection frame X and the detection frame Y in FIG. 9 overlap, the reading unit 23 removes one of the detection frames. To achieve this, the reading unit 23 calculates the IoU between the detection frames. IoU stands for Intersection over Union. If the IoU is equal to or greater than a removal threshold, the reading unit 23 removes the detection frame with the lower certainty.

[0039] (Step S205: Variable setting process) The reading unit 23 sets the number of correct digits, 32, to the variable A. The reading unit 23 sets the number of detection frames remaining after the processes of steps S203 and S204 to the variable B. The reading unit 23 sets the variable i to 1 as an initial value.

[0040] (Step S206: Detection frame number determination process) The reading unit 23 compares the variable A with the variable B. If the variable A=variable B, the reading unit 23 determines that the same number of characters as the correct number of digits, 32, have been detected, and proceeds to the process at step S207. If the variable A>variable B, the reading unit 23 determines that there is a missing character that has not been detected, and proceeds to the process at step S208. If the variable A<variable B, the reading unit 23 determines that inappropriate detection has been performed, stops the reading process, skips the estimation process, and returns the process to step S102.

[0041] (Step S207: Result setting process) The reading unit 23 sets the detected characters as the characters for each digit, starting from the specified side (the left side in the first embodiment). That is, the reading unit 23 sets the leftmost character as the first digit character. The reading unit 23 sets the second character from the left as the second digit character. This is repeated, and the character on the opposite side to the specified side (the right side in the first embodiment) is set as the character for the digit number indicated by the correct digit number 32.

[0042] (Step S208: Deficiency filling process) The reading unit 23 identifies the position of the missing character. Specifically, reading unit 23 sets the character on the specified side (left side in the first embodiment) of the detected multiple characters as the first character. Starting from i=1, reading unit 23 sets the adjacent character as the (i+1)th character if the distance between the i-th character and the adjacent character on the right side is within a reference distance. If the distance between the i-th character and the adjacent character is longer than the reference distance, reading unit 23 sets the space of the reference width next to the i-th character as a missing character as the (i+1)th character.

[0043] (Step S209: Result output process) The reading unit 23 outputs the characters read for each of the 32 correct digits to the estimation unit 24.

[0044] The defect filling process (step S208 in FIG. 8) according to the first embodiment will be described with reference to FIG. (Step S301: Average width calculation process) Reading unit 23 calculates the average width and height of the detection frames remaining after the processes of steps S203 and S204, and the average vertical coordinate (y-axis) of the upper left corner of the detection frames. Reading unit 23 sets the average width as the reference distance.

[0045] (Step S302: 1st digit setting process) The reading unit 23 sets the character on the specified side from among the detected multiple characters as the first character.

[0046] (Step S303: Frame distance calculation process) Reading unit 23 uses variable i to calculate the distance between the i-th detection window from the left and the i+1-th detection window from the left.

[0047] (Step S304: Distance determination process) The reading unit 23 determines whether the distance calculated in step S303 is within a reference distance. If the calculated distance is within the reference distance, the reading unit 23 advances the process to step S305. On the other hand, if the calculated distance is longer than the reference distance, the reading unit 23 advances the process to step S306.

[0048] (Step S305: i+1 digit setting process) The reading unit 23 sets the character in the (i+1)th detection frame from the left as the (i+1)th digit character.

[0049] (Step S306: Missing setting process) As in the case of i=1 in Fig. 11, reading unit 23 sets a space of the reference width to the right of the i-th detection frame from the left and having the same height range as the i-th detection frame from the left as a missing character in the (i+1)th digit. Here, the reference width is the average value of the widths calculated in step S301. At this time, reading unit 23 sets a detection frame in the space set as the missing character. The reading unit 23 cuts out a partial image of the space set in the (i+1)th digit from the target image data.

[0050] (Step S307: Second character recognition process) The reading unit 23 inputs the partial image cut out in step S306 into a character recognition model. A character recognition model recognizes characters shown in input image data and outputs the recognition results. The character recognition model is configured, for example, by CNN. CNN is an abbreviation for Convolutional Neural Network. In other words, unlike an object detection model, a character recognition model does not detect the position of characters. On the other hand, a character recognition model may be able to recognize characters with higher accuracy than an object detection model. When a character is recognized by the character recognition model, the reading unit 23 sets the recognized character to the (i+1)th digit. On the other hand, when a character is not recognized by the character recognition model, the reading unit 23 leaves the (i+1)th digit as a missing character.

[0051] (Step S308: Increment process) The reading unit 23 adds 1 to the variable i.

[0052] (Step S309: variable determination process) The reading unit 23 determines whether or not the variable i is equal to the variable A. If the variable i is equal to the variable A, the reading unit 23 ends the process. On the other hand, if the variable i is not equal to the variable A, the reading unit 23 advances the process to step S310.

[0053] (Step S310: Frame determination process) The reading unit 23 determines whether or not there is a detection frame to the right of the i-th detection frame from the left. If there is a detection frame, the reading unit 23 returns the process to step S303. On the other hand, if there is no detection frame, the reading unit 23 advances the process to step S311. If there is no detection frame, this means that all of the detected characters have been set as characters for any digit.

[0054] 12, the processes from step S303 to step S310 are repeatedly executed to sequentially identify defects between the leftmost detection frame and the rightmost detection frame. However, defects to the left of the leftmost detection frame and to the right of the rightmost detection frame are not identified by the processes from step S303 to step S310. Therefore, the processes from step S311 to step S317 identify defects to the left of the leftmost detection frame and to the right of the rightmost detection frame.

[0055] (Step S311: Margin calculation process) 13, the reading unit 23 calculates the distance from the leftmost detection frame to the left end of the recognition area as the first distance, and calculates the distance from the rightmost detection frame to the right end of the recognition area as the second distance.

[0056] (Step S312: Left / right determination process) The reading unit 23 determines which of the first distance and the second distance calculated in step S311 is longer. If the first distance is long, the reading unit 23 advances the process to step S313. On the other hand, if the second distance is long, the reading unit 23 advances the process to step S314. If the first distance and the second distance are the same length, the reading unit 23 advances the process to either step S313 or step S314, whichever is determined in advance. Here, if the first distance and the second distance are the same length, the reading unit 23 advances the process to step S314.

[0057] (Step S313: Left side setting process) 14, in steps S302, S305, and S306, reading unit 23 resets the character that has been set in the j-th digit of each of j=1,...,i to the character in the j+1th digit. Then, reading unit 23 sets a space of the reference width to the left of the leftmost detection frame and having the same height range as the leftmost detection frame as a missing character in the first digit. At this time, reading unit 23 sets a detection frame in the space set as the missing character. The reading unit 23 cuts out a partial image of the space set in the first digit from the target image data. When cutting out, the reading unit 23 cuts out a space specified by the average width and the average height from the average value of the top left coordinate in the vertical direction calculated in step S301.

[0058] (Step S314: Right side setting process) 15, reading unit 23 sets a space of the reference width to the right of the rightmost detection frame and having the same height range as the rightmost detection frame as a missing character in the (i+1)th digit. At this time, reading unit 23 sets a detection frame in the space set as the missing character. The reading unit 23 cuts out a partial image of the space set in the (i+1)th digit from the target image data.

[0059] (Step S315: Third character recognition process) Reading unit 23 inputs the partial image cut out in step S313 or step S314 into the character recognition model. If a character is recognized by the character recognition model, reading unit 23 sets the recognized character in place of the missing character in step S313 or step S314. On the other hand, if a character is not recognized by the character recognition model, reading unit 23 leaves it as a missing character.

[0060] (Step S316: Increment process) The reading unit 23 adds 1 to the variable i.

[0061] (Step S317: Variable determination process) The reading unit 23 determines whether or not the variable i is equal to the variable A. If the variable i is equal to the variable A, the reading unit 23 ends the process. On the other hand, if the variable i is not equal to the variable A, the reading unit 23 returns the process to step S311.

[0062] The estimation process (step S110 in FIG. 2) and the inspection process (step S113 in FIG. 2) according to the first embodiment will be described with reference to FIG. The process from step S401 to step S402 corresponds to the estimation process, and the process from step S403 to step S405 corresponds to the inspection process.

[0063] (Step S401: Read character recording process) The estimation unit 24 records in the memory 12 the characters read in step S109 for each of the 32 correct digits.

[0064] (Step S402: Character confirmation process) The estimation unit 24 regards each digit of the 32 correct digits as a target digit, and when the same character is read a reference number of times for the target digit, estimates that the character in the target digit is the read character. For example, as shown in Fig. 17, the reference number is set to 4. In this case, the estimation unit 24 refers to the characters recorded in step S401 for the target digit, and when the same character exists four times, estimates that the character in the target digit is that character. The estimation unit 24 regards the digits for which a character has been estimated as having been confirmed as characters, and excludes them from processing in the next step S402.

[0065] (Step S403: Correct answer comparison process) The checking unit 25 compares the estimated character with the characters in each of the multiple correct character strings 33 for the digits for which the character has been confirmed in step S402.

[0066] (Step S404: End determination process) The inspection unit 25 determines whether a correct answer character string 33 in which characters match for all digits indicated by the specified digits 34 has been identified, or whether a set time has elapsed. If either of these conditions is met, the inspection unit 25 advances the process to step S405. On the other hand, if neither of these conditions is met, the inspection unit 25 advances the process to step 114 in FIG. 2.

[0067] (Step S405: Correct answer determination process) When a correct answer character string 33 having matching characters for all digits indicated by the designated digits 34 is identified, the display unit 26 displays the identified correct answer character string 33 on the display device 42 and ends the process. On the other hand, when a correct answer character string 33 having matching characters for all digits indicated by the designated digits 34 is not identified, the display unit 26 displays on the display device 42 a message that the correct answer cannot be identified and ends the process.

[0068] ***Advantages of the First Embodiment*** As described above, character recognition device 10 according to the first embodiment reads characters for each digit of the 32 correct digits from a plurality of image data, and estimates the character for each digit. This makes it possible to read the character string with high accuracy even if the position where the character string is indicated cannot be accurately specified using a rotating table or the like. In addition, in the embodiment, the guide is configured to read only the character string in the recognition area indicated by the guide, which is expected to improve the accuracy of character estimation. Furthermore, the guide can be made into any shape, such as a circle or an ellipse, depending on the shape of the target object and the number of digits of the ID.

[0069] The character recognition device 10 according to the first embodiment identifies missing characters that could not be detected, thereby making it possible to read the characters digit by digit.

[0070] The character recognition device 10 according to the first embodiment performs character recognition for missing characters using a character recognition model. This may enable recognition of characters that could not be recognized using an object detection model. This improves the accuracy of character recognition, making it possible to quickly identify a character string shown on an object.

[0071] When the overlap of the detected regions is large, the character recognition device 10 according to the first embodiment removes the character with the lower confidence level. This prevents one character from being detected as being duplicated, which can cause a shift in digits.

[0072] The character recognition device 10 according to the first embodiment identifies the correct character string 33 corresponding to the estimated character string by comparing the character of each estimated digit with the character of the corresponding digit of the correct character string 33. Then, the character recognition device 10 according to the first embodiment ends acquisition of image data when the correct character string 33 corresponding to the estimated character string is identified. This prevents unnecessary repetition of processing.

[0073] The character recognition device 10 according to the first embodiment starts acquiring image data when it is recognized that an object has been set in the shooting area of ​​the camera 41 due to a change in the background image. This makes it possible to start processing easily. A first threshold value used to obtain a difference between the image data and the background image 31 in order to determine whether an object has been set in the shooting area of ​​the camera 41 is a threshold value at which the difference between the image data and the background image 31 can be confirmed. A second threshold value used to obtain a difference between the image data and the background for reading start determination in order to determine whether an object has been fixed in the shooting area of ​​the camera 41 is a threshold value at which the difference between the image data and the background image for reading start determination can be confirmed. The first threshold value and the second threshold value can also be dynamically changed.

[0074] ***Other configurations*** <Variation 1> In the first embodiment, a guide indicating a recognition area for recognizing characters is superimposed on the image data in step S103 in Fig. 2. As shown in Fig. 18, the guide may indicate not only the recognition area but also the position of the first digit. By indicating the position of the first digit with the guide, the possibility that the first digit character will be set in the correct position increases. Therefore, the possibility that the actual first digit character will be recognized as the first digit character increases.

[0075] Therefore, it may be assumed that the actual first character is recognized as the first character. In this case, the process of FIG. 10 can be simplified. Specifically, the processes from step S311 to step S313 can be omitted. This is because it is assumed that there are no characters to the left of the character initially identified as the first character. Therefore, in the processes from step S311 to step S317, missing characters are sequentially identified as being on the right side.

[0076] <Variation 2> In the first embodiment, in step S404 in Fig. 16, the inspection unit 25 determines whether a correct answer character string 33 has been identified in which the characters match for all the digits indicated by the designated digits 34. However, in step S403 in Fig. 16, the inspection unit 25 may be configured to identify, as a candidate character string, a correct answer character string 33 in which the characters match for all the digits whose characters have been confirmed in step S402. Then, in step S404 in Fig. 16, instead of determining whether a correct answer character string 33 in which the characters match for all the digits indicated by the designated digits 34 has been identified, the inspection unit 25 may determine whether the number of identified candidate character strings has decreased to one. When the process of the modified example 2 is adopted, the designation digit 34 is not necessary.

[0077] In the case where the process of the second modification is adopted, if it is determined in step S404 of FIG. 16 that the set time has elapsed, the display unit 26 may display the identified multiple candidate character strings.

[0078] <Variation 3> In the first embodiment, each functional component is realized by software. However, as a third modification, each functional component may be realized by hardware. The following describes the third modification, focusing on the differences from the first embodiment.

[0079] When each functional component is realized by hardware, the character recognition device 10 includes an electronic circuit 15 instead of the processor 11, the memory 12, and the storage 13. The electronic circuit 15 is a dedicated circuit for realizing the functions of each functional component, the memory 12, and the storage 13.

[0080] The electronic circuit 15 may be a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array. Each functional component may be realized by one electronic circuit 15, or each functional component may be distributed across multiple electronic circuits 15.

[0081] <Modification 4> As a fourth modification, some of the functional components may be realized by hardware, and other functional components may be realized by software.

[0082] The processor 11, the memory 12, the storage 13, and the electronic circuit 15 are collectively referred to as a processing circuit. In other words, the functions of the respective functional components are realized by the processing circuit.

[0083] Furthermore, the term "part" in the above description may be read as a "circuit," "step," "procedure," "processing," or "processing circuit."

[0084] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) an image acquisition unit that captures an image of an object using a camera and acquires a plurality of image data of the object; a reading unit that reads a character of each digit of a correct answer number, which is the number of digits of a character string shown on the object, from the target image data, using each of the plurality of image data acquired by the image acquisition unit as target image data; an estimation unit that estimates a character of a target digit from the character of the target digit read from each of the plurality of image data by the reading unit, with each digit of the correct digit number being a target digit; A character recognition device comprising: (Appendix 2) The image acquisition unit acquires image data of the object one by one in sequence, The reading unit sequentially reads the characters of each digit from the image data acquired by the image acquisition unit as the target image data, The estimation unit estimates the character of the target digit as a read character when the same character is read a reference number of times for the target digit. 2. A character recognition device as claimed in claim 1. (Appendix 3) The reading unit detects a plurality of characters from the target image data, sets a specified character among the detected plurality of characters as a character in a first digit, and from i=1 until the plurality of characters are set as characters in any digit, if the distance between the i-th character and an adjacent character is within a reference distance, sets the adjacent character as the i+1-th character and adds 1 to i, and if the distance between the i-th character and an adjacent character is longer than the reference distance, sets a space of a reference width next to the i-th character as a missing character as the i+1-th character and adds 1 to i, thereby reading characters in each digit. 3. A character recognition device according to claim 1 or 2. (Appendix 4) When the number of digits to which characters are set is less than the number of correct digits in a state in which the plurality of characters are set as characters for any digit, the reading unit calculates a first distance which is the distance between the most designated character of the plurality of characters and an end of the designated side in an area in which characters are to be recognized, and calculates a second distance which is the distance between the character opposite to the most designated side of the plurality of characters and an end of the area in which characters are to be recognized on the opposite side to the designated side, and when the first distance is longer than the second distance, sets a space of a reference width on the designated side of the most designated character as a missing character, and when the second distance is longer than the first distance, sets a space of a reference width on the opposite side to the designated side of the character opposite to the most designated side as a missing character. 4. A character recognition apparatus as claimed in claim 3. (Appendix 5) When the first distance is longer than the second distance, the reading unit resets the character set in the jth digit of each of j=1,...,i to the character in the j+1th digit, and sets a space of a reference width on the specified side of the character on the most specified side as a missing character to the first character, and when the second distance is longer than the first distance, sets a space of a reference width on the opposite side to the specified side of the character on the opposite side to the most specified side as a missing character to the i+1th digit. 5. A character recognition device as claimed in claim 4. (Appendix 6) The reading unit cuts out a partial image of the space determined to be the missing character from the image data, and if a character can be read from the partial image, sets the read character in place of the missing character. 6. A character recognition device according to any one of appendix 3 to 5. (Appendix 7) 7. The character recognition device according to claim 3, wherein, when an overlapping degree of an area in which two characters are detected is higher than a reference ratio, the reading unit removes the character with a lower confidence degree of the two characters and then reads the characters of each digit. (Appendix 8) The character recognition device further comprises: A checking unit that identifies a correct character string in which all designated digits designated in advance are the same as the characters estimated by the estimation unit from among a plurality of correct character strings stored in a storage device. 3. The character recognition device according to claim 2, comprising: (Appendix 9) The image acquisition unit starts acquiring the image data when it is recognized that the object has been set in the shooting area of ​​the camera due to a change in a background image, and ends acquiring the image data when the correct answer character string is identified by the inspection unit. 9. The character recognition apparatus of claim 8. (Appendix 10) The character recognition device further comprises: a checking unit that identifies, from among a plurality of correct answer character strings stored in a storage device, a correct answer character string in which all digits whose characters have been estimated by the estimation unit are the same as the estimated characters, as a candidate character string; 3. The character recognition device according to claim 2, comprising: (Appendix 11) The image acquisition unit starts acquiring the image data when it is recognized that the object has been set in the shooting area of ​​the camera due to a change in a background image, and ends acquiring the image data when the number of the candidate character strings specified by the inspection unit becomes one. 11. The character recognition apparatus of claim 10. (Appendix 12) The character recognition device further comprises: a display unit that displays a guide indicating a recognition area, which is an area in which characters are to be recognized, superimposed on the image data acquired by the image acquisition unit; 12. The character recognition device according to claim 1, comprising: (Appendix 13) The reading unit reads a character string from the recognition area portion indicated by the guide. 13. The character recognition apparatus of claim 12. (Appendix 14) The character recognition device further comprises: a read control unit that controls the reading unit to start reading when a similarity in characteristics between the image data and a background image for reading start determination, which is the most recently acquired image data, is equal to or greater than a second threshold value; 14. The character recognition device according to claim 1, comprising: (Appendix 15) The computer photographs an object using a camera to obtain a plurality of image data of the object; a computer reads, from the target image data, a character corresponding to each digit of a correct answer number, which is the number of digits in a character string shown on the target object, for each digit of the correct answer number, and A character recognition method in which a computer estimates a character in a target digit from the character in the target digit read from each of the plurality of image data, with each digit in the correct answer number being a target digit. (Appendix 16) an image acquisition process for acquiring a plurality of image data of an object by photographing the object with a camera; a reading process for reading a character of each digit of a correct answer number, which is the number of digits of a character string shown on the object, from the target image data, using each of the plurality of image data acquired by the image acquisition process as target image data; an estimation process for estimating a character of a target digit from the character of the target digit read from each of the plurality of image data by the reading process, the character being a target digit for each digit of the correct digit number; A character recognition program that causes a computer to function as a character recognition device.

[0085] The above describes the embodiments and modifications of the present disclosure. Some of these embodiments and modifications may be combined and implemented. Also, one or some of them may be partially implemented. Note that the present disclosure is not limited to the above embodiments and modifications, and various modifications are possible as necessary. [Explanation of symbols]

[0086] 10 character recognition device, 11 processor, 12 memory, 13 storage, 14 communication interface, 21 image acquisition unit, 22 reading control unit, 23 reading unit, 24 estimation unit, 25 inspection unit, 26 display unit, 31 background image, 32 number of correct digits, 33 correct string of characters, 34 specified digits, 41 camera, 42 display device.

Claims

1. an image acquisition unit that captures an image of an object using a camera and acquires a plurality of image data of the object; a reading unit that reads a character of each digit of a correct answer number, which is the number of digits of a character string shown on the object, from the target image data, using each of the plurality of image data acquired by the image acquisition unit as target image data; an estimation unit that estimates a character of a target digit from the character of the target digit read from each of the plurality of image data by the reading unit, with each digit of the correct digit number being a target digit; A character recognition device comprising:

2. The image acquisition unit acquires image data of the object one by one in sequence, The reading unit sequentially reads the characters of each digit from the image data acquired by the image acquisition unit as the target image data, The estimation unit estimates the character of the target digit as a read character when the same character is read a reference number of times for the target digit.

2. The character recognition device according to claim 1.

3. The reading unit detects a plurality of characters from the target image data, sets a specified character among the detected plurality of characters as a character in a first digit, and from i=1 until the plurality of characters are set as characters in any digit, if the distance between the i-th character and an adjacent character is within a reference distance, sets the adjacent character as the i+1-th character and adds 1 to i, and if the distance between the i-th character and an adjacent character is longer than the reference distance, sets a space of a reference width next to the i-th character as a missing character as the i+1-th character and adds 1 to i, thereby reading characters in each digit.

2. The character recognition device according to claim 1.

4. When the number of digits to which characters are set is less than the number of correct digits in a state in which the plurality of characters are set as characters for any digit, the reading unit calculates a first distance which is the distance between the most designated character of the plurality of characters and an end of the designated side in an area in which characters are to be recognized, and calculates a second distance which is the distance between the character opposite to the most designated side of the plurality of characters and an end of the area in which characters are to be recognized on the opposite side to the designated side, and when the first distance is longer than the second distance, sets a space of a reference width on the designated side of the most designated character as a missing character, and when the second distance is longer than the first distance, sets a space of a reference width on the opposite side to the designated side of the character opposite to the most designated side as a missing character.

4. The character recognition device according to claim 3.

5. When the number of digits to which characters are set is less than the number of correct digits in a state in which the plurality of characters are set as characters for any digit, the reading unit calculates a first distance which is a distance between the character on the most designated side among the plurality of characters and an end of an area for recognizing characters on the designated side, and calculates a second distance which is a distance between the character on the opposite side to the most designated side among the plurality of characters and an end of an area for recognizing characters on the opposite side to the designated side, and when the first distance is longer than the second distance, j=1 a character set in the jth digit of each of the characters 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109 ...9, 109 4. The character recognition device according to claim 3.

6. The reading unit cuts out a partial image of the space determined to be the missing character from the image data, and if a character can be read from the partial image, sets the read character in place of the missing character.

4. The character recognition device according to claim 3.

7. 4. The character recognition device according to claim 3, wherein when an overlapping degree of an area in which two characters are detected is higher than a reference ratio, the reading unit removes the character with a lower confidence degree of the two characters and then reads the characters of each digit.

8. The character recognition device further comprises: A checking unit that identifies a correct character string in which all designated digits designated in advance are the same as the characters estimated by the estimation unit from among a plurality of correct character strings stored in a storage device. The character recognition device according to claim 2 .

9. The image acquisition unit starts acquiring the image data when it is recognized that the object has been set in the shooting area of ​​the camera due to a change in a background image, and ends acquiring the image data when the correct answer character string is identified by the inspection unit.

9. The character recognition device according to claim 8.

10. The character recognition device further comprises: a checking unit that identifies, from among a plurality of correct answer character strings stored in a storage device, a correct answer character string in which all digits whose characters have been estimated by the estimation unit are the same as the estimated characters, as a candidate character string; The character recognition device according to claim 2 .

11. The image acquisition unit starts acquiring the image data when it is recognized that the object has been set in the shooting area of ​​the camera due to a change in a background image, and ends acquiring the image data when the number of the candidate character strings specified by the inspection unit becomes one. The character recognition device according to claim 10.

12. The character recognition device further comprises: a display unit that displays a guide indicating a recognition area, which is an area in which characters are to be recognized, superimposed on the image data acquired by the image acquisition unit; The character recognition device according to claim 1 .

13. The reading unit reads a character string from the recognition area portion indicated by the guide. The character recognition device according to claim 12.

14. The character recognition device further comprises: a reading control unit that controls the reading unit to start reading when a similarity in characteristics between the image data and a background image for reading start determination, which is the most recently acquired image data, is equal to or greater than a second threshold value; The character recognition device according to claim 1 .

15. The computer photographs an object using a camera to obtain a plurality of image data of the object; a computer reads, from the target image data, a character corresponding to each digit of a correct answer number, which is the number of digits of a character string shown on the target object, for each digit of the correct answer number, and A character recognition method in which a computer estimates a character in a target digit from the character in the target digit read from each of the plurality of image data, with each digit in the correct answer number being a target digit.

16. an image acquisition process for acquiring a plurality of image data of an object by photographing the object with a camera; a reading process for reading a character of each digit of a correct answer number, which is the number of digits of a character string shown on the object, from the target image data, using each of the plurality of image data acquired by the image acquisition process as target image data; an estimation process for estimating a character of a target digit from the character of the target digit read from each of the plurality of image data by the reading process, the character being a target digit for each digit of the correct digit number; A character recognition program that causes a computer to function as a character recognition device.

Images