Estimation device, estimation method, and program

Abstract

To provide an estimation device, an estimation method, and a program capable of accurately discriminating characters without using three-dimensional measurement data.SOLUTION: An estimation device 100 includes: an acquisition unit 102 that acquires image data acquired by imaging an object having a character string formed on its surface; a trimming unit 104 that sequentially performs trimming by scanning partial image data of a predetermined size from the image data in a predetermined direction; a dividing unit 106 that divides the partial image data into a plurality of pieces of divided image data of a determined size; an identifying unit 110 that performs machine learning on a combination of image data indicating a specific character and the specific character as teacher data and identifies, for each piece of divided image data, character candidates indicated in each of the plurality of pieces of divided image data and confidence degrees of the character candidates according to a learning model for predicting characters indicated in the image data; and an estimation unit 112 that estimates the character string based on a confidence degree of each piece of divided image data corresponding to each piece of the partial image data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an estimation device, an estimation method, and a program. [Background technology]

[0002] Patent Document 1 describes a method of recognizing an image of an alphanumeric character by comparing an alphanumeric character pattern image-processed based on an image signal with a plurality of alphanumeric character patterns, and outputting the recognized alphanumeric character data. Patent Documents 2 and 3 describe a method of identifying alphanumeric characters using three-dimensional measurement data. [Prior art document] [Patent documents] [Patent Document 1] Japanese Patent Application Laid-open No. 10-198804 [Patent Document 2] JP 2007-219943 A [Patent Document 3] JP 2016-33797 A Summary of the Invention [Problem to be solved by the invention]

[0003] It is desirable to be able to accurately identify characters without using three-dimensional measurement data. [Means for solving the problem]

[0004] An estimation device according to one aspect of the present invention may include an acquisition unit that acquires image data obtained by capturing an image of an object having a character string formed on its surface. The estimation device may include a trimming unit that sequentially trims partial image data of a predetermined size from the image data by scanning in a predetermined direction. The estimation device may include a division unit that divides the partial image data into a plurality of split image data of a predetermined size. The estimation device may include an identification unit that performs machine learning using combinations of image data indicating a specific character and the specific character as training data, and identifies character candidates indicated in each of the plurality of split image data and certainties of the character candidates for each of the split image data according to a learning model that predicts the character indicated in the image data. The estimation device may include an estimation unit that estimates a character string based on certainties of each of the split image data corresponding to each of the partial image data.

[0005] The estimation device may further include an image correction unit that performs image correction, including at least one of brightness, contrast, saturation, rotation, and translation, on each of the plurality of partial image data under a plurality of predetermined conditions, and generates a plurality of image-corrected partial image data from one partial image data. The division unit may divide each of the plurality of image-corrected partial image data into a plurality of split image data of a predetermined size. For each of the plurality of image-corrected split image data, the identification unit may identify character candidates and certainties of the character candidates indicated in each of the plurality of split image data according to a learning model. The estimation unit may estimate a character string based on the certainty of each of the plurality of split image data corresponding to each of the plurality of partial image data.

[0006] The estimation device may further include an image correction unit that performs image correction, including at least one of brightness, contrast, saturation, rotation, and translation, on each of the plurality of pieces of segmented image data under a plurality of predetermined conditions, and generates a plurality of image-corrected pieces of segmented image data from one piece of segmented image data.The identification unit may identify, for each of the plurality of image-corrected pieces of segmented image data, character candidates and certainties of the character candidates indicated in each of the plurality of pieces of segmented image data in accordance with a learning model.The estimation unit may estimate a character string based on the certainties of each of the plurality of pieces of segmented image data corresponding to each of the plurality of partial image data.

[0007] The identification unit may identify the character candidate with the highest average value of the certainty of the character candidates identified for each of the multiple image data segments that have been image-corrected as the final character candidate, and may identify the average value of the certainty of the character candidates as the certainty of the final character candidate.

[0008] The trimming unit may sequentially trim a plurality of partial image data from the image data of the object at a first interval in a predetermined direction, and then narrow down the image area of ​​the image data of the object to candidate areas that are likely to contain a character string based on the certainty of the plurality of divided image data of each of the plurality of partial image data identified by the identification unit, and sequentially trim the plurality of partial image data from the candidate areas at a second interval shorter than the first interval in the predetermined direction.

[0009] The surface of the object may include a circular portion, and the character string may be formed on an edge portion of the circular portion. The trimming unit may sequentially trim partial image data from the image data by scanning in a predetermined direction along an arc of the circular portion.

[0010] The estimation unit may determine that an estimation error has occurred when the estimated character string does not conform to a predetermined rule.

[0011] The estimation unit may determine that an estimation error has occurred when the certainty of at least one character candidate that constitutes the character string is equal to or less than a threshold value.

[0012] The estimation device may include a determination unit that determines whether or not retraining of the learning model is necessary based on the certainty of each character candidate of the plurality of segmented image data identified by the identification unit during a predetermined period. If retraining of the learning model is necessary, the estimation device may include a notification unit that notifies a message urging retraining of the learning model.

[0013] The character string may be formed in an uneven shape on the surface of the object.

[0014] An estimation method according to one aspect of the present invention may include acquiring image data obtained by capturing an image of an object having a character string formed on its surface. The estimation method may include sequentially trimming partial image data of a predetermined size from the image data by scanning in a predetermined direction. The estimation method may include dividing the partial image data into a plurality of split image data of a predetermined size. The estimation method may include performing machine learning using combinations of image data representing a specific character and the specific character as training data, and identifying character candidates shown in each of the plurality of split image data and certainties of the character candidates for each split image data according to a learning model that predicts the character shown in the image data. The estimation method may include estimating a character string based on certainties of each split image data corresponding to each partial image data.

[0015] A program according to one aspect of the present invention may be a program for causing a computer to function as the estimation device.

[0016] The above summary of the invention does not list all of the features of the present invention, and subcombinations of these features may also be inventions. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of a character string estimation system. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of the estimation device. [Figure 3] FIG. 10 is a diagram illustrating an example of scanning by a trimming unit. [Figure 4] FIG. 10 is a diagram illustrating image correction. [Figure 5] 10A and 10B are diagrams for explaining a method for identifying character candidates and certainty factors; [Figure 6] FIG. 10 is a diagram illustrating a method for estimating a character string. [Figure 7] 10 is a flowchart illustrating an example of a procedure for estimating a character string. [Figure 8] 10 is a flowchart illustrating an example of a procedure for determining whether or not re-learning is necessary. [Figure 9] FIG. 2 illustrates an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0018] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0019] Character strings formed in a concave and convex shape are sometimes cast and engraved on the surfaces of cast parts such as cylinder heads or cylinder blocks, cast iron pipes such as drain pipes, and sidewalls of automobile tires. There are cases where it is required to identify such character strings using image recognition.

[0020] When identifying characters formed in an uneven shape, it is conceivable to recognize the characters shown in the image data by pattern matching image data obtained by capturing an image of the characters with an imaging device such as a CCD camera with image data showing pre-registered characters. However, if the base material, i.e., the background, and the characters are similar in color, the characters can be recognized by checking the image data with the human eye, but character identification using image processing such as pattern matching may not be performed accurately.

[0021] On the other hand, by identifying characters using three-dimensional measurement data that indicates the height of the unevenness, it is possible to prevent a decrease in accuracy of identification even when the base material and characters are similar colors. However, using three-dimensional measurement data requires a three-dimensional measurement device, which increases the processing load and costs compared to using a CCD camera or other device required for two-dimensional image data.

[0022] Therefore, in this embodiment, a system is provided that can accurately distinguish characters without using three-dimensional measurement data.

[0023] FIG. 1 is a diagram showing an example of the overall configuration of a character string estimation system 10 according to this embodiment. The character string estimation system 10 includes an imaging device 30, an image collection device 50, and an estimation device 100. The imaging device 30 captures an image of an object 20 and outputs image data. The imaging device 30 includes an image sensor such as a CCD or CMOS. The object 20 has a character string 22 formed on its surface. The object 20 may be a cast part such as a cylinder head or a cylinder block, a cast iron pipe such as a drain pipe, or an automobile tire. A character string formed in an uneven pattern is cast and engraved on the surface of the object 20.

[0024] The imaging device 30 outputs image data obtained by capturing an image of the surface of the object 20. The image collection device 50 collects the image data output from the imaging device 30 and stores it in a memory. The image collection device 50 may store the image data in a database present on a network. The estimation device 100 uses the image data collected by the image collection device 50 to estimate a character string formed on the surface of the object 20.

[0025] 2 shows an example of functional blocks of the estimation device 100. The estimation device 100 includes an acquisition unit 102, a cropping unit 104, a division unit 106, an image correction unit 108, an identification unit 110, an estimation unit 112, a determination unit 114, and a notification unit 116.

[0026] The acquisition unit 102 acquires image data obtained by capturing an image of an object 20 having a character string formed on its surface via the image collection device 50. The trimming unit 104 sequentially trims partial image data of a predetermined size from the image data by scanning the image data in a predetermined direction. The trimming unit 104 may sequentially trim partial image data from the image data by scanning the image data along the direction in which the character string is arranged. The predetermined size corresponds to the size of the character string within the image area of ​​the image data.

[0027] The portion of the object 20 on which the character string 22 is formed may be a circular portion or a ring-shaped portion, such as a flange portion of a pipe. The character string 22 may be formed on the edge portion of the circular portion. The trimming unit 104 may sequentially trim a plurality of partial image data from the image data by scanning in a predetermined direction along the arc of the circular portion.

[0028] 3 shows an example of scanning by the trimming unit 104. The trimming unit 104 detects a circular portion of the object 20 from the image area of ​​the image data. The trimming unit 104 may detect the circular portion of the object 20 from the image area by, for example, edge detection. The trimming unit 104 scans a trimming frame 24 of a predetermined size at least once around the axis at constant small displacement angles in the circumferential direction around the axis based on the radius r of the circle of the detected circular portion and the coordinate information of the center P, and sequentially trims partial image data from the image data.

[0029] The trimming unit 104 may perform polar coordinate conversion using the radius r of the detected circular portion and coordinate information of the center P, and may scan the trimming frame 24 one or more times around the center coordinates in the circumferential direction at regular small displacement angles around the axis, thereby sequentially trimming partial image data from the image data. The trimming unit 104 may use a Hough transform to identify the radius r of the circular portion and coordinate information of the center P, and may perform polar coordinate conversion using the radius r of the circular portion and coordinate information of the center P, and may scan the trimming frame 24 one or more times around the center coordinates in the circumferential direction at regular small displacement angles around the axis, thereby sequentially trimming partial image data from the image data.

[0030] When the trimming unit 104 scans the trimming frame 24 at every small displacement angle, it is preferable that the identification unit 110 identify characters using the partial image data that has been trimmed when the center of gravity of the trimming frame 24 and the center of gravity of the character string to be identified are close to each other. Furthermore, the brightness, contrast, saturation, etc. of the image data captured by the imaging device 30 may change due to the influence of ambient light, etc., which may reduce the accuracy of character identification.

[0031] Therefore, the image correction unit 108 may perform image correction including at least one of brightness, contrast, saturation, rotation, and translation on each of the plurality of partial image data under a plurality of predetermined conditions, and generate a plurality of image-corrected partial image data from one partial image data. The image correction unit 108 may perform image correction on each of the plurality of partial image data under a plurality of conditions in which at least one parameter of brightness, contrast, saturation, rotation, and translation is randomly changed, and generate a plurality of image-corrected partial image data from one partial image data.

[0032] The dividing unit 106 divides the image-corrected partial image data into a plurality of divided image data pieces of predetermined sizes, which correspond to the predetermined sizes of characters expected in the image area of ​​the image data.

[0033] 4, the image correction unit 108 performs image correction on one partial image data 210 under multiple conditions in which at least one parameter of brightness, contrast, saturation, rotation, and translation is randomly changed, thereby generating multiple partial image data 210. The division unit 106 divides each of the multiple partial image data 210 into multiple divided image data 202 of a predetermined size.

[0034] The identification unit 110 performs machine learning using a combination of image data indicating a specific character and the specific character as training data, and identifies character candidates indicated in each of the multiple pieces of segmented image data 202 and the confidence levels of the character candidates for each piece of segmented image data 202 according to a learning model that predicts characters indicated in the image data. The learning model may be a deep learning model. The identification unit 110 may associate the identified character candidates with the confidence levels of the character candidates and register them in table 250. The identification unit 110 may identify the confidence levels using the output of a Softmax function, for example. The identification unit 110 may identify the confidence levels using the similarity, for example, COS similarity, between the feature vectors of the image data indicating the characters that serve as training data and the feature vectors of the segmented image data.

[0035] The image correction unit 108 may perform image correction individually on each of the divided image data divided by the dividing unit 106. That is, the image correction unit 108 may perform image correction including at least one of brightness, contrast, saturation, rotation, and translation on each of the plurality of divided image data 202 under a plurality of predetermined conditions, and generate a plurality of image-corrected divided image data 202 from one divided image data 202. Then, the identification unit 110 may identify, for each of the plurality of image-corrected divided image data 202, character candidates indicated in each of the plurality of divided image data 202 and the certainty of the character candidates according to the learning model.

[0036] Even cast characters can be distinguished by the human eye in most cases. Therefore, by using a learning method based on a deep learning model that is similar to the character recognition means of humans, the identification unit 110 can identify characters with high accuracy.

[0037] For example, as shown in Fig. 5, after performing image correction, the identification unit 110 identifies a confidence level for each character candidate for each piece of divided image data 202 after image correction. Then, the identification unit 110 identifies the candidate character with the highest average confidence level among the candidate characters identified for each piece of divided image data 202 as the final candidate character. In the example shown in Fig. 5, the character candidate "E" has the highest average confidence level, so the identification unit 110 identifies the final character candidate as "E" and the confidence level as "0.73," which is the average confidence level of the character candidate "E."

[0038] In this way, the specification unit 110 specifies the candidate character with the highest average confidence level among the candidate characters specified for each of the multiple pieces of divided image data 202 as the final candidate character, and specifies the average confidence level of the character candidates as the confidence level of the final candidate character. This makes it possible to prevent an increase in the erroneous recognition rate due to the influence of factors such as an adjacent character being cut off or external disturbances.

[0039] Actual data often results in an unbalanced data set in which the frequency of appearance of each character varies. In such cases, it is common to use a learning method such as focal loss to correct bias in the output of a deep learning model due to data imbalance. However, when a character with a distinctive shape, such as the character "8," is cut off, it is more likely to be misclassified as the cut-off character than as a character with a simple shape, such as the character "1." On the other hand, according to this embodiment, the probability that the identification unit 110 will misclassify a candidate character can be reduced by using the average value of the classification results and confidence factors for each of the multiple image data segments that have been image-corrected.

[0040] The estimation unit 112 estimates a character string based on the certainty factor of each divided image data 202 corresponding to each partial image data 210. As shown in FIG. 6, the estimation unit 112 derives a total certainty factor by summing the certainty factors of character candidates in each divided image data of each partial image data 210. That is, the estimation unit 112 sums the certainty factors of character candidates in multiple divided image data 202 divided from one partial image data 210 to derive one total certainty factor for one partial image data 210. Then, the estimation unit 112 selects the partial image data with the highest total certainty factor to estimate a character string. For example, in the example shown in FIG. 6, the estimation unit 112 selects the partial image data 210 with a certainty factor of "4.1" and estimates the character string as "ABC223."

[0041] In this way, the estimation unit 112 identifies a certainty factor for each piece of divided image data and estimates the character string based on the total certainty factor. This allows the estimation unit 112 to estimate the character string 22 using the partial image data 210 in which the center of gravity 120 of the character string 22 and the center of gravity 214 of the trimmed partial image data 210 overlap most closely.

[0042] When estimating a character string based on a predetermined rule, such as a serial number, the estimation unit 112 may finalize the estimated character string if the estimated character string is based on the predetermined rule. If the estimated character string is not based on the predetermined rule, the estimation unit 112 may determine that an estimation error has occurred, i.e., that the character string is unreadable. Furthermore, the estimation unit 112 may determine that an estimation error has occurred if the certainty of at least one character candidate constituting the character string is equal to or less than a threshold.

[0043] The trimming unit 104 may sequentially trim multiple partial image data 210 from the image data of the object 20 at a first interval in a predetermined direction, and then narrow down the image region of the image data of the object 20 to candidate regions likely to contain the character string 22 based on the confidence levels of the multiple divided image data 202 of each of the multiple partial image data 210 identified by the identification unit 110. The trimming unit 104 may then sequentially trim the multiple partial image data 210 in the predetermined direction at a second interval shorter than the first interval for each candidate region. Within the image region of the image data, regions containing character strings are limited. Therefore, the trimming unit 104 may first trim the image region at a relatively large interval, and then narrow down candidate regions likely containing character strings based on the confidence levels of a small number of divided image data 202. The trimming unit 104 may then further trim the candidate regions at a relatively small interval. The estimation unit 112 may finally estimate the character string 22 based on the confidence levels of the divided image data 202 of each partial image data 210 within the candidate region. This reduces the processing load for character string estimation.

[0044] In addition, when the impact of processing load is small, for example, when the scanning range of the entire image area is small, the trimming unit 104 does not perform trimming twice, but trims the image data at an interval equivalent to the second interval described above in the first trimming stage, and the identification unit 110 identifies character candidates and their certainty based on the multiple partial image data 210 obtained by trimming, and the estimation unit 112 may estimate the character string 22.

[0045] Furthermore, due to a change in the environment in which the object 20 is imaged, the error between the image data captured by the imaging device 30 and the image data used as training data may increase, which may result in a decrease in the accuracy of the candidate character strings identified by the identification unit 110 in accordance with the learning model and their certainty. In other words, there is a possibility that the certainty identified by the identification unit 110 may continually decrease. In such a case, it is preferable to retrain the learning model using new training data.

[0046] Therefore, the determination unit 114 determines whether or not retraining of the learning model is necessary based on the confidence of each character candidate of the plurality of segmented image data 202 identified by the identification unit 110 during a predetermined period. The determination unit 114 may determine that retraining of the learning model is necessary when the average value of the total confidence of each character candidate of the plurality of segmented image data 202 identified by the identification unit 110 during a predetermined period is equal to or less than a predetermined reference value. If retraining of the learning model is necessary, the notification unit 116 notifies the user of a message urging the user to retrain the learning model. The notification unit 116 may notify the user of the message urging the user to retrain the learning model by email or the like. The notification unit 116 may display a message urging the user to retrain the learning model on a display of a management device that manages the character string estimation system 10.

[0047] FIG. 7 is a flowchart showing an example of a procedure for estimating a character string.

[0048] The trimming unit 104 trims partial image data at a first interval from an image region of the image data including the object 20 (S100). The image correction unit 108 performs image correction on each of the plurality of partial image data, including at least one of brightness, contrast, saturation, rotation, and translation, under a plurality of predetermined conditions, to generate a plurality of image-corrected partial image data from one partial image data (S102). The division unit 106 divides each of the plurality of image-corrected partial image data 210 into a plurality of divided image data 202 (S104).

[0049] The identification unit 110 identifies character candidates shown in each of the multiple pieces of segmented image data 202 and the certainty of the character candidates for each piece of segmented image data 202 according to the learning model (S106). The trimming unit 104 determines a candidate area from the image area of ​​the image data based on the certainty of the segmented image data 202 (S108). The certainty of the segmented image data 202 in an area that does not include a character string is likely to be 0 or close to 0. On the other hand, the certainty of the segmented image data 202 in an area that includes at least a part of a character string is likely to be higher than 0, for example, 0.50 or higher. Therefore, the trimming unit 104 narrows down the candidate areas to areas where the certainty of the segmented image data 202 is relatively high, and sequentially trims the multiple partial image data 210 from the candidate area in a predetermined direction at second intervals that are shorter than the first intervals (S110).

[0050] The image correction unit 108 performs image correction on one piece of partial image data under multiple conditions in which at least one parameter of brightness, contrast, saturation, rotation, and translation is randomly changed, thereby generating multiple pieces of partial image data (S112). The division unit 106 divides each of the multiple pieces of image data 210 that have undergone image correction into multiple pieces of divided image data 202 (S114). Next, the identification unit 110 identifies character candidates shown in each of the multiple pieces of divided image data 202 and the certainty of the character candidates for each piece of divided image data 202 in accordance with the learning model (S116). The estimation unit 112 estimates a character string based on the certainty of each piece of divided image data 202 corresponding to each piece of partial image data 210 (S118).

[0051] As described above, the trimming unit 104 narrows down the candidate areas where a character string is likely to exist, and the estimation unit 112 finally estimates the character string 22. This reduces the processing load for estimating the character string. The identification unit 110 identifies characters according to a trained learning model, so characters can be recognized with high accuracy without using a three-dimensional measurement device or a measurement table for measurement to recognize character strings formed in an uneven shape.

[0052] Furthermore, the identification unit 110 identifies characters according to a learning model using multiple pieces of divided image data after image correction under multiple conditions. This makes it easier to identify character candidates with high confidence, even when the identification unit 110 identifies characters according to a learning model using image data that differs in brightness, etc. from the image data used as training data due to differences in the imaging environment.

[0053] 8 is a flowchart showing an example of a procedure for determining whether or not re-learning is required. The determination unit 114 derives an average value of the total certainty of the certainty of each character candidate of a plurality of divided image data 202 identified by the identification unit 110 during a predetermined period (S200). Next, the determination unit 114 determines whether or not the average value of the total certainty is equal to or less than a reference value (S202). If the average value of the total certainty is equal to or less than the reference value, the notification unit 116 notifies a message prompting re-learning of the learning model if re-learning of the learning model is required (S204).

[0054] As described above, when the confidence level decreases, a message is displayed urging the user to retrain the learning model, so the user can immediately understand the need for retraining.

[0055] 9 illustrates an example of a computer 1200 in which aspects of the present invention may be embodied, in whole or in part. A program installed on the computer 1200 may cause the computer 1200 to perform operations associated with an apparatus according to an embodiment of the present invention or to function as one or more “parts” of the apparatus. Alternatively, the program may cause the computer 1200 to execute the operations or one or more “parts.” The program may cause the computer 1200 to execute a process or steps of a process according to an embodiment of the present invention. Such a program may be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0056] The computer 1200 according to this embodiment includes a CPU 1212 and a RAM 1214, which are interconnected by a host controller 1210. The computer 1200 also includes a communication interface 1222 and an input / output unit, which are connected to the host controller 1210 via an input / output controller 1220. The computer 1200 also includes a ROM 1230. The CPU 1212 operates according to programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit.

[0057] The communication interface 1222 communicates with other electronic devices via a network. A hard disk drive may store programs and data used by the CPU 1212 in the computer 1200. The ROM 1230 stores a boot program executed by the computer 1200 upon activation and / or programs dependent on the computer's hardware. The programs may be provided via a computer-readable recording medium such as a CD-ROM, a USB memory, or an IC card, or via a network. The programs may be installed in the RAM 1214 or the ROM 1230, which are also examples of computer-readable recording media, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200 and establishes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0058] For example, when communication is performed between computer 1200 and an external device, CPU 1212 may execute a communication program loaded into RAM 1214 and instruct communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 1212, communication interface 1222 reads transmission data stored in a transmission buffer area provided in RAM 1214 or a recording medium such as a USB memory, and transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0059] The CPU 1212 may also cause all or a necessary portion of a file or database stored on an external recording medium such as a USB memory to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write the processed data back to the external recording medium.

[0060] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0061] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0062] A computer-readable medium may include any tangible device capable of storing instructions that are executed by an appropriate device. As a result, the computer-readable medium with instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (RTM) disc, memory stick, integrated circuit card, etc.

[0063] The computer-readable instructions may include either source code or object code written in any combination of one or more programming languages. The source code or object code includes conventional procedural programming languages. The conventional procedural programming languages ​​may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and the “C” programming language or similar programming languages. The computer-readable instructions may be provided to a processor or programmable circuitry of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus locally or over a wide-area network (WAN) such as a local area network (LAN), the Internet, etc. The processor or programmable circuitry may execute the computer-readable instructions to create means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0064] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0065] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0066] 10 String Prediction System 20 Object 22 Strings 24 Trimming Frame 30 Imaging device 50 Image acquisition device 100 Estimator 102 Acquisition Department 104 Trimming section 106 Split section 108 Image Correction Unit 110 Specific section 112 Estimation Department 114 Judgment section 116 Notification Department 120 Center of gravity 202 divided image data 210 Partial Image Data 250 tables 1210 host controller 1212 CPU 1214 RAM 1220 Input / Output Controller 1222 communication interface 1230 ROM

Claims

1. an acquisition unit that acquires image data obtained by capturing an image of an object having a character string formed on its surface; a trimming unit that sequentially trims partial image data of a predetermined size from the image data by scanning the partial image data in a predetermined direction; a dividing unit that divides the partial image data into a plurality of divided image data of a predetermined size; an identification unit that performs machine learning using a combination of image data showing a specific character and the specific character as training data, and identifies character candidates shown in each of the plurality of divided image data and certainty of the character candidates for each divided image data according to a learning model that predicts characters shown in the image data; an estimation unit that estimates the character string based on the certainty of each divided image data corresponding to each partial image data; Equipped with the surface of the object includes a circular portion, and the character string is formed on an edge portion of the circular portion; The trimming unit sequentially trims the partial image data from the image data by scanning in the predetermined direction along the arc of the circular portion.

2. an image correction unit that performs image correction including at least one of brightness, contrast, saturation, rotation, and translation on each of the plurality of partial image data under a plurality of predetermined conditions, and generates a plurality of image-corrected partial image data from one partial image data; the dividing unit divides each of the plurality of image data segments that have been subjected to image correction into a plurality of image data segments each having the predetermined size; the identification unit identifies character candidates indicated in each of the plurality of image data segments that have undergone image correction and certainties of the character candidates according to the learning model; The estimation device according to claim 1 , wherein the estimation unit estimates the character string based on a certainty factor of each of the plurality of divided image data corresponding to each of the plurality of partial image data.

3. an image correction unit that performs image correction including at least one of brightness, contrast, saturation, rotation, and translation on each of the plurality of pieces of divided image data under a plurality of predetermined conditions, and generates a plurality of pieces of image-corrected divided image data from one piece of divided image data; the identification unit identifies character candidates indicated in each of the plurality of image data segments that have undergone image correction and certainties of the character candidates according to the learning model; The estimation device according to claim 1 , wherein the estimation unit estimates the character string based on a certainty factor of each of the plurality of divided image data corresponding to each of the plurality of partial image data.

4. 4. The estimation device according to claim 2, wherein the identification unit identifies the character candidate with the highest average value of certainty among the character candidates identified for each of the plurality of image data segments after image correction as the final character candidate, and identifies the average value of the certainty of the character candidates as the certainty of the final character candidate.

5. 5. The estimation device according to claim 1, wherein the trimming unit sequentially trims a plurality of partial image data from the image data of the object at a first interval in the predetermined direction, and then narrows down candidate areas that are likely to contain character strings from the image area of ​​the image data of the object based on the certainty of the plurality of divided image data of each of the plurality of partial image data identified by the identification unit, and sequentially trims a plurality of partial image data from the candidate areas at a second interval shorter than the first interval in the predetermined direction.

6. The estimation device according to claim 1 , wherein the estimation unit determines that an estimation error has occurred when the estimated character string does not conform to a predetermined rule.

7. The estimation device according to claim 1 , wherein the estimation unit determines that an estimation error has occurred when the certainty factor of at least one of the character candidates constituting the character string is equal to or less than a threshold value.

8. a determination unit that determines whether or not re-learning of the learning model is necessary based on the certainty of each character candidate of the plurality of pieces of divided image data identified by the identification unit during a predetermined period; a notification unit that notifies a message prompting the re-learning of the learning model when the re-learning of the learning model is necessary; The estimation device according to claim 1 , further comprising:

9. The estimation device according to claim 1 , wherein the character string is formed in an uneven shape on the surface of the object.

10. acquiring image data obtained by capturing an image of an object having a character string formed on its surface; a step of sequentially trimming partial image data of a predetermined size from the image data by scanning in a predetermined direction; Dividing the partial image data into a plurality of divided image data of a predetermined size; a step of performing machine learning using a combination of image data showing a specific character and the specific character as training data, and identifying character candidates shown in each of the plurality of divided image data and the certainty of the character candidates for each divided image data according to a learning model that predicts characters shown in the image data; a step of estimating the character string based on the certainty of each divided image data corresponding to each partial image data; Equipped with the surface of the object includes a circular portion, and the character string is formed on an edge portion of the circular portion; The estimation method, wherein the step of sequentially trimming includes the step of sequentially trimming the partial image data from the image data by scanning in the predetermined direction along the arc of the circular portion.

11. A program for causing a computer to function as the estimation device according to any one of claims 1 to 9.

Images