Character recognition device, character recognition method, and character recognition program

Abstract

To provide a character recognition device capable of accurately reading a character string engraved on an object without a rotary stand or the like.SOLUTION: An image acquisition part 21 acquires multiple pieces of image data of an object by imaging the object by a camera 41. A reading part 23 reads a character in each digit from the image data of the object, in each digit of a correct digit number being a digit number of the character string engraved on the object. An estimation part 24 estimates the character in the digit to be estimated as the character which is read, when the same character is read by reference times in the digit to be estimated, in which the respective digits in the correct digit number are the digits to be estimated.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a technology for reading a character string such as an ID displayed on an object such as a wafer. ID is an abbreviation for IDentifier. [Background technology]

[0002] When processing objects such as wafers, visual inspection is required to ensure the object is correct. However, errors can easily occur during visual inspection due to factors such as small text on the object or text being difficult to read due to reflections from lighting. As a result, there is a risk of processing the wrong object. In addition, visual inspection requires the allocation of additional workers, which takes time and effort.

[0003] Patent Document 1 describes a method in which a wafer is placed on a turntable, and the position where the ID is written is identified based on the position of the notch in the wafer. In Patent Document 1, the wafer is rotated by the turntable to move the position where the ID is written into the shooting area of ​​a camera, and the ID is read. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-261501 [Patent Document 2] Japanese Patent Application Publication No. 2018-173818 [Patent Document 3] Japanese Patent Application Publication No. 2018-018366 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, the position where the ID is written is identified and the ID position is moved into the camera's shooting area using a turntable, thereby enabling the ID to be read with high accuracy. However, in Patent Document 1, the wafer needs to be placed on the turntable, which requires a lot of rework if the wrong wafer is placed on the turntable. Furthermore, in Patent Document 1, the wafer is rotated using the turntable, so the wafer comes into contact with the turntable, which is a piece of equipment, and there is a possibility that the wafer will be scratched. Furthermore, if a scratch is discovered on the wafer after it has been placed on the turntable, it is impossible to determine whether the scratch occurred before or after the wafer was placed on the turntable, making it difficult to determine responsibility. An object of the present disclosure is to enable accurate reading of character strings displayed on an object without using a rotating table or the like. [Means for solving the problem]

[0006] A character recognition device according to the present disclosure includes: an image acquisition unit that captures an image of an object using a camera and acquires a plurality of image data of the object one by one in sequence; a reading unit that sequentially sets each of the plurality of image data acquired by the image acquisition unit as target image data, and reads a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, from the target image data; an estimation unit that estimates, with each digit of the correct digit number as a target digit, the character of the target digit as a read character when the same character has been read for the target digit a reference number of times by the reading unit; a checking unit that identifies a correct character string in which all designated digits designated in advance are the same as the characters estimated by the estimation unit from among a plurality of correct character strings stored in a storage device; Equipped with. [Effects of the Invention]

[0007] In the present disclosure, characters are read for each digit of the correct number of digits from multiple image data, and characters are estimated for each digit. This makes it possible to read character strings with high accuracy even if the position of the character string cannot be accurately identified using a rotating table or the like. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of a character recognition device 10 according to a first embodiment. [Figure 2] 3 is a flowchart of the overall processing of the character recognition device 10 according to the first embodiment. [Figure 3] 3 is a diagram showing an example of a correct number of digits 32, a correct character string 33, and designated digits 34 according to the first embodiment. FIG. [Figure 4] FIG. 2 is an explanatory diagram of a method for using the character recognition device 10 according to the first embodiment. [Figure 5] 4A to 4C are explanatory diagrams of a screen display process and a recognition area display process according to the first embodiment. [Figure 6] FIG. 3 is an explanatory diagram of an estimated image display process according to the first embodiment. [Figure 7] FIG. 4 is an explanatory diagram of an estimated character display process according to the first embodiment. [Figure 8] 4 is a flowchart of a reading process according to the first embodiment. [Figure 9] FIG. 3 is an explanatory diagram of overlap removal processing according to the first embodiment. [Figure 10] 10 is a flowchart of a defect filling process according to the first embodiment. [Figure 11] FIG. 3 is an explanatory diagram of a loss setting process according to the first embodiment. [Figure 12] FIG. 3 is an explanatory diagram of a defect filling process according to the first embodiment. [Figure 13] FIG. 3 is an explanatory diagram of a margin calculation process according to the first embodiment. [Figure 14] FIG. 4 is an explanatory diagram of a left-side setting process according to the first embodiment. [Figure 15] FIG. 4 is an explanatory diagram of a right-side setting process according to the first embodiment. [Figure 16] FIG. 3 is an explanatory diagram of an estimation process and an inspection process according to the first embodiment. [Figure 17] FIG. 4 is an explanatory diagram of the character confirmation process according to the first embodiment. [Figure 18] FIG. 10 is an explanatory diagram of a guide according to a first modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] Embodiment 1 ***Configuration Description*** The configuration of a character recognition device 10 according to the first embodiment will be described with reference to FIG. The character recognition device 10 is a computer. The character recognition device 10 includes hardware such as a processor 11, a memory 12, a storage 13, and a communication interface 14. The processor 11 is connected to other hardware via signal lines and controls the other hardware.

[0010] The processor 11 is an IC that performs processing. IC stands for Integrated Circuit. Specific examples of the processor 11 include a CPU, a DSP, and a GPU. CPU stands for Central Processing Unit. DSP stands for Digital Signal Processor. GPU stands for Graphics Processing Unit.

[0011] The memory 12 is a storage device that temporarily stores data. Specific examples of the memory 12 include SRAM and DRAM. SRAM stands for Static Random Access Memory. DRAM stands for Dynamic Random Access Memory.

[0012] The storage 13 is a storage device that stores data. A specific example of the storage 13 is an HDD. HDD is an abbreviation for Hard Disk Drive. The storage 13 may also be a portable recording medium such as an SD (registered trademark) memory card, CompactFlash (registered trademark), NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. SD is an abbreviation for Secure Digital. DVD is an abbreviation for Digital Versatile Disk.

[0013] The communication interface 14 is an interface for communicating with an external device. Specific examples of the interface 14 include ports for Ethernet (registered trademark), USB, and HDMI (registered trademark). USB stands for Universal Serial Bus. HDMI stands for High-Definition Multimedia Interface.

[0014] The character recognition device 10 is connected to a camera 41 and a display device 42 via a communication interface 14. The camera 41 is an imaging device for capturing an image of an object. The display device 42 is a display such as an LCD for displaying the reading results, etc. LCD is an abbreviation for Liquid Crystal Display.

[0015] Character recognition device 10 includes, as functional components, an image acquisition unit 21, a reading control unit 22, a reading unit 23, an estimation unit 24, an inspection unit 25, and a display unit 26. The functions of each functional component of character recognition device 10 are realized by software. Storage 13 stores programs that realize the functions of each functional component of character recognition device 10. These programs are loaded into memory 12 by processor 11 and executed by processor 11. As a result, the functions of each functional component of character recognition device 10 are realized.

[0016] The storage 13 stores a background image 31, a correct answer digit number 32, a plurality of correct answer character strings 33, and a specified digit number 34.

[0017] 1 shows only one processor 11. However, there may be a plurality of processors 11, and the plurality of processors 11 may cooperate to execute programs that realize the respective functions.

[0018] ***Explanation of Operation*** The operation of the character recognition device 10 according to the first embodiment will be described with reference to FIGS. The operation procedure of character recognition device 10 according to embodiment 1 corresponds to the character recognition method according to embodiment 1. Furthermore, the program that realizes the operation of character recognition device 10 according to embodiment 1 corresponds to the character recognition program according to embodiment 1.

[0019] In the first embodiment, it is assumed that processing is performed on a group of objects, such as several tens of objects, according to their IDs. Here, it is assumed that each object is assigned a different ID with the same number of digits.

[0020] The overall processing of the character recognition device 10 according to the first embodiment will be described with reference to FIG. As a prerequisite for the processing of FIG. 2, a background image 31, a correct answer digit number 32, a plurality of correct answer character strings 33, and designated digits 34 are set in the storage 13 by the user. The background image 31 is image data obtained by capturing an image with the camera 41 when no object, such as a wafer, is in the capturing area of ​​the camera 41. The correct digit count 32 is the number of digits of the character string displayed on the object. In the first embodiment, the correct digit count 32 is the number of digits of the ID displayed on the object. The correct character string 33 is the character string displayed on the object. In the first embodiment, the correct character string 33 is the ID displayed on the object, and each of the IDs displayed on all objects is set as the correct character string 33. The designated digits 34 are one or more digits that must be estimated. In the first embodiment, the designated digits 34 are one or more digits that are required to uniquely identify the ID. For example, suppose that an ID assigned to a group of objects is seven digits long and the first two digits are the same. In this case, the last five digits of the seven digits are designated as the designated digits 34. The correct digit number 32, the correct character string 33, and the designated digits 34 may be set as one piece of data, as shown in FIG. 3. In FIG. 3, for each digit of the number of digits indicated by the correct digit number 32, the correct character and whether or not the digit is a required digit are indicated by the designated digits 34. The first two digits (the first digit) The first and second digits are the common part of the ID, so the required result is False (not required), and the last 5 digits are required to identify the ID, so they are defined as True (required).

[0021] In the first embodiment, as shown in Fig. 4, a worker manually moves an object into the shooting area of ​​camera 41, and reads an ID, which is a string of characters displayed on the object, from image data of the object obtained by camera 41. The worker wears protective gloves or the like to prevent the object from being scratched, and then holds the object. If the read ID is the intended ID, the object is ready to be processed.

[0022] (Step S101: Background reading process) The reading control unit 22 reads the background image 31 from the storage 13 .

[0023] (Step S102: First image acquisition process) The image acquisition unit 21 acquires image data obtained by capturing an image of a capture area using the camera 41. It is assumed that the resolution and the like of the camera 41 are specified in advance.

[0024] (Step S103: Image display processing) 5A, the display unit 26 superimposes a guide indicating a recognition area for recognizing characters on the image data acquired in step S102 and displays the superimposed guide on the display device 42. The (x, y) coordinates of the upper left and the (x, y) coordinates of the lower right of the guide are assumed to be specified in advance.

[0025] (Step S104: Recognition area display process) As shown in FIG. 5B, the display unit 26 displays on the display device 42 an image of the recognition area portion indicated by the guide, out of the image data acquired in step S102.

[0026] (Step S105: Background determination process) The reading control unit 22 compares the image data acquired in step S102 with the background image 31 read in step S101. The reading control unit 22 determines whether the difference between the image data and the background image 31 is equal to or greater than a first threshold value. If the difference is equal to or greater than the first threshold, the reading control unit 22 determines that an object has entered the photographing area of ​​the camera 41, and proceeds to step S106. On the other hand, if the difference is less than the first threshold, the reading control unit 22 determines that an object has not entered the photographing area of ​​the camera 41, and returns the process to step S102. When the process is returned to step S102, new image data is acquired in step S102.

[0027] (Step S106: Previous image setting process) The reading control unit 22 sets the most recently acquired image data as the background image for reading start determination. The most recently acquired image data is the image data acquired in step S102 or the image data acquired in step S107, which will be described later.

[0028] (Step S107: Second image acquisition process) The image acquisition unit 21 acquires image data obtained by photographing an imaging area with the camera 41 .

[0029] (Step S108: Previous image determination process) The reading control unit 22 compares the image data acquired in step S107 with the background image for reading start determination set in step S106. The reading control unit 22 determines whether the similarity of the features between the image data and the background image for reading start determination is less than a second threshold value. If the similarity of the features is less than the second threshold, the reading control unit 22 determines that the position of the ID, which is a character string, is being aligned with the recognition area, and returns the process to step S106. On the other hand, if the similarity of the features is equal to or greater than the second threshold, the reading control unit 22 determines that the position of the ID, which is a character string, has been aligned with the recognition area, and proceeds to step S109.

[0030] (Step S109: Reading process) The reading unit 23 sets the most recently acquired image data as the target image data, and reads the character of each digit of the 32 correct digits from the target image data. If the process of step S109 is executed for the first time, the most recently acquired image data is the image data acquired in step S107. If the process of step S109 is executed for the second time or later, the most recently acquired image data is the image data acquired in step S114.

[0031] (Step S110: Estimation process) The estimation unit 24 estimates the character of the target digit from the characters of the target digit that have been read in step S109 so far, with each digit of the 32 correct digits being the target digit.

[0032] (Step S111: Estimated image display process) As shown in Fig. 6(A), the display unit 26 displays the image data used for reading in step S109. Also, as shown in Fig. 6(B), the display unit 26 displays on the display device 42 an image of the recognition area portion indicated by the guide, out of the image data used for reading in step S109.

[0033] (Step S112: Estimated character display process) 7, the display unit 26 displays the characters estimated in step S110. Here, only the characters for digits that have been estimated are displayed, and characters for digits that have not been estimated are displayed as blanks.

[0034] (Step S113: Inspection process) The checking unit 25 checks whether the guess is complete using the correct answer character string 33 and the specified digits 34. If the estimation is complete, the inspection unit 25 ends the process. On the other hand, if the estimation is not complete, the inspection unit 25 returns the process to step S102. In step S113, the checking unit 25 checks whether the estimation of the specified digits 34 for which the required item is defined as True has been completed in the correct character string 33. Note that the checking unit 25 may be configured to check the entire number of correct digits 32 including the required item as False.

[0035] The reading process (step S109 in FIG. 2) according to the first embodiment will be described with reference to FIG. (Step S201: First extraction process) The reading unit 23 cuts out a partial image of the input size for the object detection model from the target image data, using the center position of the recognition area as a reference. For example, the reading unit 23 cuts out a partial image of the input size whose center position is the center position of the recognition area. The object detection model is a trained model that detects characters using image data as input. The object detection model has a set input size, which is the size of the image data to be input.

[0036] (Step S202: First character recognition process) The reading unit 23 inputs the partial image cut out in step S201 into the object detection model. The object detection model detects characters from input image data and outputs the position of the detected characters and the recognition result of the detected characters. A rectangular detection frame surrounding the character is output. The character recognition result indicates which character is detected and the degree of certainty of the recognition.

[0037] (Step S203: Out-of-range removal process) The reading unit 23 removes characters outside the recognition area from among the characters detected by the object detection model in step S202. Specifically, the reading unit 23 removes characters whose detection frames are outside the recognition area by a reference proportion or more. This leaves only characters within the recognition area.

[0038] (Step S204: Duplicate removal process) If the degree of overlap between the regions where the two characters are detected is higher than a reference rate, the reading unit 23 removes the character with the lower degree of certainty of the two characters. This will be explained in detail with reference to FIG. 9. The detection frames output from the object detection model may overlap. The detection frames may be slightly misaligned, causing the detection frames of adjacent characters to overlap, or a single character may be detected multiple times, causing the detection frames to overlap. When the detection frames of adjacent characters overlap, the reading unit 23 leaves the detection frames as they are. On the other hand, when a single character is detected multiple times, the reading unit 23 leaves only one detection frame and removes the remaining detection frames. In other words, for the overlapping portion of detection frame U and detection frame V in FIG. 9, the detection frames remain as they are. On the other hand, for the overlapping portion of detection frame X and detection frame Y in FIG. 9, the reading unit 23 removes one of the detection frames. To achieve this, the reading unit 23 calculates the IoU between the detection windows. IoU stands for Intersection over Union. If the IoU is equal to or greater than a removal threshold, the reading unit 23 removes the detection window with the lower confidence.

[0039] (Step S205: Variable setting process) The reading unit 23 sets the number of correct digits, 32, to variable A. The reading unit 23 sets the number of detection frames remaining after the processes of steps S203 and S204 to variable B. The reading unit 23 sets 1 as the initial value to variable i.

[0040] (Step S206: Detection frame number determination process) The reading unit 23 compares variable A with variable B. If variable A=variable B, the reading unit 23 determines that the same number of characters as the correct digit number 32 have been detected, and proceeds with the process to step S207. If variable A>variable B, the reading unit 23 determines that there are missing characters that have not been detected, and proceeds with the process to step S208. If variable A<variable B, the reading unit 23 determines that inappropriate detection has been performed, stops the reading process, skips the estimation process, and returns the process to step S102.

[0041] (Step S207: Result setting process) The reading unit 23 sets the detected characters as the characters for each digit, starting from the specified side (the left side in the first embodiment). That is, the reading unit 23 sets the leftmost character as the first digit character. The reading unit 23 sets the second character from the left as the second digit character. This is repeated, and the character on the opposite side from the specified side (the right side in the first embodiment) is set as the character for the digit number indicated by the correct digit number 32.

[0042] (Step S208: Deficiency filling process) The reading unit 23 identifies the position of the missing character. Specifically, the reading unit 23 sets the character on the specified side (the left side in the first embodiment) of the detected characters as the character in the first digit. If the distance between the i-th character and the adjacent character on the right is within the reference distance, the reading unit 23 sets the adjacent character as the (i+1)th character, starting from i=1. If the distance between the i-th character and the adjacent character is longer than the reference distance, the reading unit 23 sets the space of the reference width next to the i-th character as a missing character and sets it as the (i+1)th character. Set to the character.

[0043] (Step S209: Result output process) The reading unit 23 outputs the characters read for each of the 32 correct digits to the estimation unit 24.

[0044] The defect filling process (step S208 in FIG. 8) according to the first embodiment will be described with reference to FIG. (Step S301: Average width calculation process) The reading unit 23 calculates the average width and height of the detection frames remaining after the processes of steps S203 and S204, and the average vertical coordinate (y-axis) of the upper left corner of the detection frames. The reading unit 23 sets the average width as the reference distance.

[0045] (Step S302: First digit setting process) The reading unit 23 sets the character on the specified side among the detected characters as the first character.

[0046] (Step S303: Inter-frame distance calculation process) Using the variable i, the reader 23 calculates the distance between the ith detection box from the left and the (i+1)th detection box from the left.

[0047] (Step S304: Distance determination process) The reading unit 23 determines whether the distance calculated in step S303 is within a reference distance. If the calculated distance is within the reference distance, the reading unit 23 proceeds to step S305. On the other hand, if the calculated distance is longer than the reference distance, the reading unit 23 proceeds to step S306.

[0048] (Step S305: i+1 digit setting process) The reading unit 23 sets the character in the (i+1)th detection frame from the left as the (i+1)th character.

[0049] (Step S306: Missing setting process) As in the case of i=1 in Figure 11, reading unit 23 sets a space of the reference width to the right of the ith detection frame from the left and having the same height range as the ith detection frame from the left as a missing character in the (i+1)th digit. Here, the reference width is the average value of the widths calculated in step S301. At this time, reading unit 23 sets a detection frame in the space set as a missing character. The reading unit 23 cuts out a partial image of the space set at the (i+1)th digit from the target image data.

[0050] (Step S307: Second character recognition process) The reading unit 23 inputs the partial image cut out in step S306 into the character recognition model. A character recognition model recognizes characters shown in input image data and outputs the recognition results. A character recognition model is configured, for example, by a CNN. CNN stands for Convolutional Neural Network. In other words, unlike an object detection model, a character recognition model does not detect the position of characters. However, a character recognition model may be able to recognize characters with higher accuracy than an object detection model. When a character is recognized by the character recognition model, the reading unit 23 sets the recognized character to the (i+1)th digit. On the other hand, when a character is not recognized by the character recognition model, the reading unit 23 leaves the (i+1)th digit as a missing character.

[0051] (Step S308: Increment process) The reading unit 23 adds 1 to the variable i.

[0052] (Step S309: Variable determination process) The reading unit 23 determines whether or not the variable i is equal to the variable A. If the variable i is equal to the variable A, the reading unit 23 ends the process. On the other hand, if the variable i is not equal to the variable A, the reading unit 23 advances the process to step S310.

[0053] (Step S310: Frame determination process) The reading unit 23 determines whether or not there is a detection frame to the right of the i-th detection frame from the left. If there is a detection frame, the reading unit 23 returns the process to step S303. On the other hand, if there is no detection frame, the reading unit 23 proceeds to step S311. If there is no detection frame, this means that all of the detected characters have been set as characters for any digit.

[0054] 12, the processes of steps S303 to S310 are repeatedly executed, thereby sequentially identifying defects between the leftmost and rightmost detection frames. However, defects to the left of the leftmost detection frame and to the right of the rightmost detection frame are not identified by the processes of steps S303 to S310. Therefore, defects to the left of the leftmost detection frame and to the right of the rightmost detection frame are identified by the processes of steps S311 to S317.

[0055] (Step S311: Margin calculation process) 13, the reading unit 23 calculates the distance from the leftmost detection frame to the left edge of the recognition area as the first distance, and calculates the distance from the rightmost detection frame to the right edge of the recognition area as the second distance.

[0056] (Step S312: Left / right determination process) The reading unit 23 determines which of the first distance and the second distance calculated in step S311 is longer. If the first distance is long, reading unit 23 proceeds to step S313. On the other hand, if the second distance is long, reading unit 23 proceeds to step S314. Note that if the first distance and the second distance are the same length, reading unit 23 proceeds to step S313 or step S314, whichever is predetermined. Here, reading unit 23 proceeds to step S314 if the first distance and the second distance are the same length.

[0057] (Step S313: Left side setting process) 14, in steps S302, S305, and S306, reading unit 23 resets the character that has been set in the jth digit for each of j=1,...,i to the character in the j+1th digit. Then, reading unit 23 sets a space of the reference width to the left of the leftmost detection frame and having the same height range as the leftmost detection frame as a missing character in the first digit. At this time, reading unit 23 sets a detection frame in the space that has been set as the missing character. The reading unit 23 cuts out a partial image of the space set in the first digit from the target image data. When cutting out, the reading unit 23 cuts out a space specified by the average width and average height from the average value of the top left coordinate in the vertical direction calculated in step S301.

[0058] (Step S314: Right side setting process) 15, reading unit 23 sets a space of the reference width to the right of the rightmost detection frame and having the same height range as the rightmost detection frame as a missing character in the (i+1)th digit. At this time, reading unit 23 sets a detection frame in the space set as the missing character. The reading unit 23 extracts a partial image of a space set at the (i+1)th digit from the target image data. break the ice.

[0059] (Step S315: Third character recognition process) Reading unit 23 inputs the partial image cut out in step S313 or step S314 into the character recognition model. If a character is recognized by the character recognition model, reading unit 23 sets the recognized character in place of the missing character in step S313 or step S314. On the other hand, if a character is not recognized by the character recognition model, reading unit 23 leaves the character as a missing character.

[0060] (Step S316: Increment process) The reading unit 23 adds 1 to the variable i.

[0061] (Step S317: Variable determination process) The reading unit 23 determines whether or not the variable i is equal to the variable A. If the variable i is equal to the variable A, the reading unit 23 ends the process. On the other hand, if the variable i is not equal to the variable A, the reading unit 23 returns the process to step S311.

[0062] The estimation process (step S110 in FIG. 2) and the inspection process (step S113 in FIG. 2) according to the first embodiment will be described with reference to FIG. The processes from step S401 to step S402 correspond to the estimation process, and the processes from step S403 to step S405 correspond to the inspection process.

[0063] (Step S401: Read character recording process) The estimation unit 24 records the characters read in step S109 in the memory 12 for each of the 32 correct digits.

[0064] (Step S402: Character confirmation process) The estimation unit 24 sets each digit of the 32 correct digits as a target digit, and when the same character is read a reference number of times for the target digit, estimates that the character in the target digit is the read character. For example, as shown in Fig. 17, the reference number of times is set to 4. In this case, the estimation unit 24 refers to the characters recorded in step S401 for the target digit, and when the same character exists four times, estimates that the character in the target digit is that character. The estimation unit 24 determines that the digits for which characters have been estimated have been confirmed as characters, and excludes them from the processing target in the next step S402.

[0065] (Step S403: Correct answer comparison process) The checking unit 25 compares the estimated character with the characters in each of the multiple correct character strings 33 for the digits for which the character has been confirmed in step S402.

[0066] (Step S404: End determination process) The checking unit 25 determines whether a correct answer character string 33 has been identified, in which characters match for all digits indicated by the specified digits 34, or whether a set time has elapsed. If either of these conditions is met, the checking unit 25 proceeds to step S405. On the other hand, if neither of these conditions is met, the checking unit 25 proceeds to step 114 in FIG. 2.

[0067] (Step S405: Correct answer determination process) If a correct answer character string 33 is identified in which the characters match for all the digits indicated by the designated digits 34, the display unit 26 displays the identified correct answer character string 33 on the display device 42 and ends the process. On the other hand, if a correct answer character string 33 in which the characters match for all the digits indicated by the designated digits 34 has not been identified, the display unit 26 displays on the display device 42 a message that the correct answer cannot be identified and ends the process. Finish.

[0068] ***Effects of the First Embodiment*** As described above, character recognition device 10 according to the first embodiment reads characters for each digit of the 32 correct digits from a plurality of image data and estimates the character for each digit. This makes it possible to read character strings with high accuracy even if the position where the character string is displayed cannot be accurately identified using a rotating table or the like. In addition, in the embodiment, the system is configured to read only the character string in the recognition area indicated by the guide, which is expected to improve the accuracy of character estimation. Furthermore, the guide can be made into any shape, such as a circle or ellipse, depending on the shape of the target object and the number of digits of the ID.

[0069] The character recognition device 10 according to the first embodiment identifies missing characters that could not be detected, thereby making it possible to read the characters digit by digit.

[0070] The character recognition device 10 according to the first embodiment performs character recognition for missing characters using a character recognition model. This may enable recognition of characters that could not be recognized using an object detection model. This improves the accuracy of character recognition, enabling the character string displayed on the target object to be identified quickly.

[0071] When the overlap of detected regions is large, the character recognition device 10 according to the first embodiment removes the character with the lower confidence level. This prevents a single character from being detected as being duplicated, resulting in a shift in digits.

[0072] The character recognition device 10 according to the first embodiment identifies the correct character string 33 corresponding to the estimated character string by comparing the character of each estimated digit with the character of the corresponding digit in the correct character string 33. Then, the character recognition device 10 according to the first embodiment stops acquiring image data when the correct character string 33 corresponding to the estimated character string is identified. This prevents unnecessary repetition of processing.

[0073] The character recognition device 10 according to the first embodiment starts acquiring image data when it recognizes that an object has been set in the image capture area of ​​the camera 41 due to a change in the background image. This allows for easy initiation of processing. The first threshold used to calculate the difference between the image data and the background image 31 in order to determine whether the object has been set in the image capture area of ​​the camera 41 is a threshold at which the difference between the image data and the background image 31 can be confirmed. Furthermore, the second threshold used to calculate the difference between the image data and the background image for determining whether the object has been fixed in the image capture area of ​​the camera 41 is a threshold at which the difference between the image data and the background image for determining whether the object has been fixed in the image capture area of ​​the camera 41 can be confirmed. The first and second thresholds can also be dynamically changed.

[0074] ***Other Configurations*** <Variation 1> In the first embodiment, a guide indicating a recognition area for recognizing characters is superimposed on the image data in step S103 of Fig. 2. As shown in Fig. 18, the guide may indicate not only the recognition area but also the position of the first digit. By indicating the position of the first digit with the guide, the first digit character is more likely to be set in the correct position. Therefore, the actual first digit character is more likely to be recognized as the first digit character.

[0075] Therefore, it may be assumed that the actual first character is recognized as the first character. In this case, the processing in FIG. 10 can be simplified. Specifically, the processing from step S311 to step S313 can be omitted. This is because it is assumed that there is no character to the left of the character initially identified as the first character. Therefore, In the processing from step S311 to step S317, missing characters are identified in order on the right side.

[0076] <Variation 2> In the first embodiment, in step S404 of Fig. 16, the checking unit 25 determines whether a correct answer character string 33 has been identified, in which the characters match for all the digits indicated by the designated digits 34. However, in step S403 of Fig. 16, the checking unit 25 may be configured to identify, as candidate character strings, correct answer character strings 33 in which the characters match for all the digits whose characters have been confirmed in step S402. Then, in step S404 of Fig. 16, instead of determining whether a correct answer character string 33 in which the characters match for all the digits indicated by the designated digits 34 has been identified, the checking unit 25 may determine whether only one candidate character string has been identified. When the processing of the modified example 2 is adopted, the designation digit 34 is not necessary.

[0077] In the case where the process of the second modification is adopted, if it is determined in step S404 of FIG. 16 that the set time has elapsed, the display unit 26 may display the identified plurality of candidate character strings.

[0078] <Variation 3> In the first embodiment, each functional component is realized by software. However, as a third modification, each functional component may be realized by hardware. The following describes the differences between the first embodiment and the third modification.

[0079] When each functional component is realized by hardware, the character recognition device 10 includes an electronic circuit 15 instead of the processor 11, the memory 12, and the storage 13. The electronic circuit 15 is a dedicated circuit for realizing the functions of each functional component, the memory 12, and the storage 13.

[0080] The electronic circuit 15 may be a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be realized by one electronic circuit 15, or each functional component may be realized by distributing it among a plurality of electronic circuits 15.

[0081] <Variation 4> As a fourth modification, some of the functional components may be realized by hardware, and other functional components may be realized by software.

[0082] The processor 11, memory 12, storage 13, and electronic circuit 15 are collectively referred to as a processing circuit. In other words, the functions of the respective functional components are realized by the processing circuit.

[0083] Furthermore, the term "unit" in the above description may be read as a "circuit," "step," "procedure," "process," or "processing circuit."

[0084] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) an image acquisition unit that photographs an object using a camera and acquires a plurality of image data of the object; Each of the plurality of image data acquired by the image acquisition unit is treated as target image data. a reading unit that reads a character of each digit of a correct answer number of digits, which is the number of digits of a character string shown on the object, from the image data of the object; an estimation unit that estimates a character of a target digit from the character of the target digit read from each of the plurality of image data by the reading unit, with each digit of the correct digit number being a target digit; A character recognition device comprising: (Appendix 2) the image acquisition unit acquires image data of the object one by one in sequence; the reading unit sequentially reads the character of each digit from the image data acquired by the image acquisition unit as the target image data; The estimation unit estimates the character of the target digit as the read character when the same character is read a reference number of times for the target digit. 2. The character recognition device of claim 1. (Appendix 3) The reading unit detects a plurality of characters from the target image data, sets a specified character among the detected plurality of characters as the first character, and from i=1 until the plurality of characters are set as characters of any one of the digits, if the distance between the i-th character and an adjacent character is within a reference distance, sets the adjacent character as the i+1-th character and adds 1 to i, and if the distance between the i-th character and an adjacent character is longer than the reference distance, sets a space of a reference width next to the i-th character as a missing character as the i+1-th character and adds 1 to i, thereby reading the characters of each digit. 3. The character recognition device according to claim 1 or 2. (Appendix 4) When the number of digits to which the characters are set is less than the number of correct digits in a state in which the plurality of characters are set as characters for any digit, the reading unit calculates a first distance, which is the distance between the character closest to the specified side among the plurality of characters and an end of the specified side in an area in which characters are recognized, and calculates a second distance, which is the distance between the character closest to the specified side among the plurality of characters and an end of the area in which characters are recognized on the opposite side from the specified side, and when the first distance is longer than the second distance, sets a space of a reference width on the specified side of the character closest to the specified side as a missing character, and when the second distance is longer than the first distance, sets a space of a reference width on the opposite side from the specified side of the character opposite to the specified side as a missing character. 4. The character recognition device of claim 3. (Appendix 5) When the first distance is longer than the second distance, the reading unit resets the character set in the jth digit of each of j=1,...,i to the character in the j+1th digit, and sets a space of a standard width on the specified side of the character on the most specified side as a missing character in the first digit, and when the second distance is longer than the first distance, sets a space of a standard width on the opposite side of the specified side of the character opposite to the most specified side as a missing character in the i+1th digit. 5. The character recognition device of claim 4. (Appendix 6) The reading unit cuts out a partial image of the space that is determined to be the missing character from the image data, and if a character can be read from the partial image, sets the read character in place of the missing character. 6. The character recognition device according to any one of appendix 3 to 5. (Appendix 7) 7. The character recognition device according to any one of appendices 3 to 6, wherein, when the degree of overlap of an area in which two characters are detected is higher than a reference rate, the reading unit removes the character with the lower confidence of the two characters and then reads the characters for each digit. (Appendix 8) The character recognition device further comprises: All the designated digits specified in advance from a plurality of correct answer character strings stored in a storage device are A verification unit that identifies a correct character string that is the same as the character estimated by the estimation unit. 3. The character recognition device according to claim 2, comprising: (Appendix 9) The image acquisition unit starts acquiring the image data when it recognizes that the object has been set in the camera's shooting area due to a change in the background image, and ends acquiring the image data when the correct answer character string is identified by the inspection unit. 9. The character recognition device of claim 8. (Appendix 10) The character recognition device further comprises: a checking unit that identifies, from a plurality of correct answer character strings stored in a storage device, a correct answer character string in which all digits of characters estimated by the estimation unit are the same as the estimated characters, as a candidate character string; 3. The character recognition device according to claim 2, comprising: (Appendix 11) The image acquisition unit starts acquiring the image data when it recognizes that the object has been set in the camera's shooting area due to a change in the background image, and ends acquiring the image data when the number of candidate character strings identified by the inspection unit becomes one. 11. The character recognition apparatus of claim 10. (Appendix 12) The character recognition device further comprises: a display unit that displays a guide indicating a recognition area, which is an area where characters are to be recognized, superimposed on the image data acquired by the image acquisition unit; 12. The character recognition device according to claim 1, comprising: (Appendix 13) The reading unit reads a character string from the recognition area indicated by the guide. 13. The character recognition apparatus of claim 12. (Appendix 14) The character recognition device further comprises: a reading control unit that controls the reading unit to start reading when a similarity in features between the image data and a background image for reading start determination, which is image data most recently acquired, is equal to or greater than a second threshold value; 14. The character recognition device according to any one of claims 1 to 13, comprising: (Appendix 15) a computer photographing an object with a camera and acquiring a plurality of image data of the object; a computer, which regards each of the plurality of image data as target image data, reads a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, from the target image data; A character recognition method in which a computer estimates a character in a target digit from characters in the target digit read from each of the plurality of image data, with each digit in the correct number of digits being a target digit. (Appendix 16) an image acquisition process of photographing an object using a camera and acquiring a plurality of image data of the object; a reading process in which, using each of the plurality of image data acquired by the image acquisition process as target image data, a character at each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, is read from the target image data; an estimation process in which each digit of the correct digit number is set as a target digit and a character of the target digit is estimated from the character of the target digit read from each of the plurality of image data by the reading process; A character recognition program that enables a computer to function as a character recognition device.

[0085] The embodiments and modifications of the present disclosure have been described above. Some of these embodiments and modifications may be combined and implemented. Also, one or more of them may be implemented partially. Note that the present disclosure is not limited to the above embodiments and modifications, and various modifications are possible as needed. [Explanation of symbols]

[0086] 10 character recognition device, 11 processor, 12 memory, 13 storage, 14 communication interface, 21 image acquisition unit, 22 reading control unit, 23 reading unit, 24 estimation unit, 25 inspection unit, 26 display unit, 31 background image, 32 number of correct digits, 33 correct string of characters, 34 specified digits, 41 camera, 42 display device.

Claims

1. an image acquisition unit that captures an image of an object using a camera and acquires a plurality of image data of the object one by one in sequence; a reading unit that sequentially sets each of the plurality of image data acquired by the image acquisition unit as target image data, and reads a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, from the target image data; an estimation unit that estimates, with each digit of the correct digit number as a target digit, the character of the target digit as a read character when the same character has been read for the target digit a reference number of times by the reading unit; a checking unit that identifies a correct character string in which all designated digits designated in advance are the same as the characters estimated by the estimation unit from among a plurality of correct character strings stored in a storage device; Equipped with The image acquisition unit starts acquiring the image data when it recognizes that the object has been set in the camera's shooting area due to a change in the background image, and stops acquiring the image data when the correct character string is identified by the inspection unit.

2. an image acquisition unit that captures an image of an object using a camera and acquires a plurality of image data of the object one by one in sequence; a reading unit that sequentially sets each of the plurality of image data acquired by the image acquisition unit as target image data, and reads a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, from the target image data; an estimation unit that estimates, with each digit of the correct digit number as a target digit, the character of the target digit as a read character when the same character has been read for the target digit a reference number of times by the reading unit; a checking unit that identifies a correct character string from a plurality of correct character strings stored in a storage device, the correct character string having all designated digits designated in advance being the same as the characters estimated by the estimation unit; a reading control unit that controls the reading unit to start reading when a similarity in features between the image data and a background image for reading start determination, which is image data most recently acquired, is equal to or greater than a second threshold; A character recognition device comprising:

3. an image acquisition unit that captures an image of an object using a camera and acquires a plurality of image data of the object one by one in sequence; a reading unit that sequentially sets each of the plurality of image data acquired by the image acquisition unit as target image data, and reads a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, from the target image data; an estimation unit that estimates, with each digit of the correct digit number as a target digit, the character of the target digit as a read character when the same character has been read for the target digit a reference number of times by the reading unit; a checking unit that identifies, from a plurality of correct answer character strings stored in a storage device, a correct answer character string in which all digits of characters estimated by the estimation unit are the same as the estimated characters, as a candidate character string; Equipped with The image acquisition unit starts acquiring the image data when it recognizes that the object has been set in the camera's shooting area due to a change in the background image, and stops acquiring the image data when the number of candidate character strings identified by the inspection unit becomes one.

4. an image acquisition unit that captures an image of an object using a camera and acquires a plurality of image data of the object one by one in sequence; a reading unit that sequentially sets each of the plurality of image data acquired by the image acquisition unit as target image data, and reads a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, from the target image data; an estimation unit that estimates, with each digit of the correct digit number as a target digit, the character of the target digit as a read character when the same character has been read for the target digit a reference number of times by the reading unit; a checking unit that identifies, from a plurality of correct answer character strings stored in a storage device, a correct answer character string in which all digits of characters estimated by the estimation unit are the same as the estimated characters, as a candidate character string; a reading control unit that controls the reading unit to start reading when a similarity in features between the image data and a background image for reading start determination, which is image data most recently acquired, is equal to or greater than a second threshold; A character recognition device comprising:

5. The computer photographs an object using a camera and sequentially acquires a plurality of image data of the object one by one; The computer sequentially sets each of the plurality of image data as target image data, and reads the character of each digit of the correct number of digits, which is the number of digits of the character string shown on the target object, from the target image data; the computer sets each digit of the correct number of digits as a target digit, and when the same character is read for the target digit a reference number of times, estimates the character for the target digit as the read character; The computer identifies a correct character string from a plurality of correct character strings stored in a storage device, in which all of the designated digits designated in advance are the same as the estimated characters; A character recognition method in which a computer starts acquiring the image data when it recognizes that the object has been set in the camera's shooting area due to a change in the background image, and stops acquiring the image data when the correct character string is identified.

6. The computer photographs an object using a camera and sequentially acquires a plurality of image data of the object one by one; The computer sequentially sets each of the plurality of image data as target image data, and reads the character of each digit of the correct number of digits, which is the number of digits of the character string shown on the target object, from the target image data; the computer sets each digit of the correct number of digits as a target digit, and when the same character is read for the target digit a reference number of times, estimates the character for the target digit as the read character; The computer identifies a correct character string from a plurality of correct character strings stored in a storage device, in which all of the designated digits designated in advance are the same as the estimated characters; A character recognition method in which a computer controls to start reading when the similarity of features between image data and a background image for determining whether to start reading, which is image data most recently obtained, is equal to or greater than a second threshold.

7. The computer photographs an object using a camera and sequentially acquires a plurality of image data of the object one by one; The computer sequentially sets each of the plurality of image data as target image data, and reads the character of each digit of the correct number of digits, which is the number of digits of the character string shown on the target object, from the target image data; the computer sets each digit of the correct number of digits as a target digit, and when the same character is read for the target digit a reference number of times, estimates the character for the target digit as the read character; The computer identifies, from a plurality of correct answer character strings stored in the storage device, a correct answer character string in which all digits of the estimated characters are the same as the estimated characters, as a candidate character string; A character recognition method in which a computer starts acquiring the image data when it recognizes that the object has been set in the camera's shooting area due to a change in the background image, and stops acquiring the image data when only one candidate character string has been identified.

8. The computer photographs an object using a camera and sequentially acquires a plurality of image data of the object one by one; The computer sequentially sets each of the plurality of image data as target image data, and reads the character of each digit of the correct number of digits, which is the number of digits of the character string shown on the target object, from the target image data; the computer sets each digit of the correct number of digits as a target digit, and when the same character is read for the target digit a reference number of times, estimates the character for the target digit as the read character; The computer identifies, from a plurality of correct answer character strings stored in the storage device, a correct answer character string in which all digits of the estimated characters are the same as the estimated characters, as a candidate character string; A character recognition method in which a computer controls to start reading when the similarity of features between image data and a background image for determining whether to start reading, which is image data most recently obtained, is equal to or greater than a second threshold.

9. an image acquisition process of photographing an object using a camera and acquiring a plurality of image data of the object one by one; a reading process in which, using each of the plurality of image data acquired by the image acquisition process as target image data in turn, a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, is read from the target image data; an estimation process in which, when the same character is read for each digit of the correct number of digits as a target digit by the reading process, the character of the target digit is estimated as a read character; a checking process for identifying a correct character string in which all designated digits specified in advance are the same as the characters estimated by the estimation process from among a plurality of correct character strings stored in a storage device; The computer functions as a character recognition device that performs the following: In the image acquisition process, a character recognition program starts acquiring the image data when it recognizes that the object has been set in the camera's shooting area due to a change in the background image, and ends acquiring the image data when the correct character string is identified by the inspection process.

10. an image acquisition process of photographing an object using a camera and acquiring a plurality of image data of the object one by one; a reading process in which, using each of the plurality of image data acquired by the image acquisition process as target image data in turn, a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, is read from the target image data; an estimation process in which, when the same character is read for each digit of the correct number of digits as a target digit by the reading process, the character of the target digit is estimated as a read character; a checking process for identifying a correct character string in which all designated digits specified in advance are the same as the characters estimated by the estimation process from among a plurality of correct character strings stored in a storage device; a reading control process that controls the reading process to start reading when a similarity of features between the image data and a background image for reading start determination, which is image data most recently acquired, is equal to or greater than a second threshold value; and A character recognition program that enables a computer to function as a character recognition device.

11. an image acquisition process of photographing an object using a camera and acquiring a plurality of image data of the object one by one; a reading process in which, using each of the plurality of image data acquired by the image acquisition process as target image data in turn, a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, is read from the target image data; an estimation process in which, when the same character is read for each digit of the correct number of digits as a target digit by the reading process, the character of the target digit is estimated as a read character; a checking process for identifying, from a plurality of correct answer character strings stored in a storage device, correct answer character strings in which all digits of characters estimated by the estimation process are the same as the estimated characters, as candidate character strings; The computer functions as a character recognition device that performs the following: In the image acquisition process, a character recognition program starts acquiring the image data when it recognizes that the object has been set in the camera's shooting area due to a change in the background image, and ends acquiring the image data when the number of candidate character strings identified by the inspection process becomes one.

12. an image acquisition process of photographing an object using a camera and acquiring a plurality of image data of the object one by one; a reading process in which, using each of the plurality of image data acquired by the image acquisition process as target image data in turn, a character of each digit of a correct number of digits, which is the number of digits of a character string shown on the target object, is read from the target image data; an estimation process in which, when the same character is read for each digit of the correct number of digits as a target digit by the reading process, the character of the target digit is estimated as a read character; a checking process for identifying, from a plurality of correct answer character strings stored in a storage device, correct answer character strings in which all digits of characters estimated by the estimation process are the same as the estimated characters, as candidate character strings; a reading control process that controls the reading process to start reading when a similarity of features between the image data and a background image for reading start determination, which is image data most recently acquired, is equal to or greater than a second threshold value; and A character recognition program that enables a computer to function as a character recognition device.

Images