Information processing device, estimation method, and estimation program

Abstract

This information processing device (1) comprises: a difference curve generation unit (104) that generates a difference curve indicating the difference between a luminance accumulation curve generated by accumulating luminance values of pixels of an image obtained by imaging a character string and a smoothing curve obtained by smoothing the luminance accumulation curve; and an estimation unit (107) that estimates a range in which the character string appears in the image on the basis of the difference curve.

Description

Information processing device, estimation method, and estimation program

[0001] The present invention relates to an information processing device or the like that estimates an area in which a character string appears in an image.

[0002] Character recognition techniques for recognizing character strings in an image have been known for some time. To improve recognition accuracy, character recognition may involve estimating the range in an image in which a character string appears, and then extracting the range to perform character recognition. For example, in the character recognition method described in Patent Document 1 below, an image in which a character string appears is first binarized. This character recognition method then detects the upper and lower edges of the character string from the binarized image to extract a line of character string, and then identifies the spaces between characters in the extracted line of character string to extract and recognize each individual character.

[0003] Japanese Patent Application Publication No. 2008-250754

[0004] However, the above-mentioned conventional techniques have a problem in that it is difficult to correctly extract character strings when the contrast of the image is low. This is because when a low-contrast image is binarized, non-character parts are often displayed as black pixels, and conversely, character parts are displayed as white pixels.

[0005] An object of one aspect of the present invention is to realize an information processing device or the like that can accurately estimate an area in which a character string appears even in an image with low contrast.

[0006] In order to solve the above problem, an information processing device according to one aspect of the present invention includes: a brightness accumulation curve generated by accumulating the brightness values ​​of each pixel arranged in a second direction perpendicular to a first direction in which the character string is arranged for an image captured of a character string on a target object; a difference curve generation unit that generates a difference curve showing the difference between the brightness accumulation curve and a smoothed curve obtained by smoothing the brightness accumulation curve; and an estimation unit that estimates the range in which the character string appears in the image based on the difference curve.

[0007] In order to solve the above problem, an estimation method according to one aspect of the present invention is a method for estimating a range of a character string executed by one or more information processing devices, and includes: a difference curve generation process for generating a difference curve showing the difference between a brightness accumulation curve generated by accumulating the brightness values ​​of each pixel arranged in a second direction perpendicular to a first direction in which the character string is arranged, along a first direction in which the character string is arranged, and a smoothed curve obtained by smoothing the brightness accumulation curve; and an estimation process for estimating the range in which the character string appears in the image based on the difference curve.

[0008] The information processing device according to each aspect of the present invention may be realized by a computer. In this case, a control program that causes the computer to operate as each part (software element) of the information processing device to realize the information processing device on a computer, and a computer-readable recording medium on which the control program is recorded, also fall within the scope of the present invention.

[0009] According to one aspect of the present invention, it is possible to accurately estimate the range in which character strings appear even in an image with low contrast.

[0010] 8 is a diagram illustrating an overview of an information processing system according to an embodiment of the present invention; FIG. 9 is a block diagram illustrating an example of the configuration of an information processing device according to an embodiment of the present invention; FIG. 10 is a diagram illustrating an example of an image of an ophthalmic lens captured by a camera; FIG. 11 is an image on which edge detection processing has been performed; FIG. 12 is an image showing the contour of an ophthalmic lens; FIG. 13 is an image converted into a band shape; FIG. 14 is an example of a luminance accumulation curve; FIG. 15 is an example of a luminance accumulation curve, a smoothing curve, and a difference curve; FIG. 16 is an enlarged view of a portion of FIG. 10; FIG. 17 is a diagram illustrating a comparison of a difference curve with a threshold; FIG. 18 is a graph illustrating the results of a quantization processing; FIG. 19 is a diagram illustrating the division of character blocks; FIG. 19 is an example of a graph plotting inflection points; FIG. 19 is a diagram illustrating an example of a method for extracting character strings; FIG. 19 is a diagram illustrating an example of a method for enhancing contrast; FIG. 19 is a flowchart illustrating an example of processing executed by an information processing device according to an embodiment of the present invention; FIG. 19 is an example of an image converted into a band shape according to a modified example of the present invention; FIG. 19 is an example of a cluster classification result.

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0012] (Outline of Information Processing System) An outline of an information processing system 100 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an outline of the information processing system 100. The information processing system 100 is a system that can be used to support processing related to an ophthalmic lens 3, and includes an information processing device 1, a camera 2, and an ophthalmic lens 3. The ophthalmic lens 3 in this embodiment is, for example, a corneal orthodontic contact lens. However, this is not limited to this. The ophthalmic lens 3 may be a soft contact lens or a hard contact lens.

[0013] The installation location of the camera 2 is not particularly limited, and for example, the camera 2 may be installed in a facility that manufactures the ophthalmic lens 3. The camera 2 has an imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and captures an image of the ophthalmic lens 3 from above. The image captured by the camera 2 is transmitted to the information processing device 1. Note that, although the camera 2 and the information processing device 1 are shown as separate devices in FIG. 1 , this is not limiting. The camera 2 may be formed so as to be integrated with the information processing device 1. In other words, the information processing device 1 may have the camera 2.

[0014] The information processing device 1 performs multiple image processing operations, which will be described later, on the image captured by the camera 2, and estimates the range in which the character string engraved on the ophthalmic lens 3 is captured. The information processing device 1 then recognizes the character string within the estimated range and outputs the recognized character string. An example of the configuration of the information processing device 1 and an example of multiple image processing operations will be described below.

[0015] (Configuration Example of Information Processing Device 1) FIG. 2 is a block diagram showing an example of the configuration of the information processing device 1. As shown in FIG.

[0016] 2 , the information processing device 1 includes a control unit 10 that controls each unit of the information processing device 1 and a storage unit 11 that stores various data used by the information processing device 1. The information processing device 1 also includes a communication unit 12 that enables the information processing device 1 to communicate with other devices (e.g., the camera 2), an input unit 13 that accepts various data input to the information processing device 1, and an output unit 14 that enables the information processing device 1 to output various data. The control unit 10 also includes an image conversion unit 101, an integral curve generation unit 102, a smoothing curve generation unit 103, a difference curve generation unit 104, a candidate detection unit 105, a classification unit 106, an estimation unit 107, an image reconstruction unit 108, a character recognition unit 109, and an output control unit 110. The storage unit 11 stores an image 111 of the ophthalmic lens 3 captured by the camera 2.

[0017] The information processing device 1 may be configured as a general-purpose computer. When the information processing device 1 is configured as a general-purpose computer, the control unit 10 may be configured to include a processor, a read-only memory (ROM), a random access memory (RAM), etc. The processor can read programs from the ROM and execute various programs using the RAM as a working area. The storage unit 11 may be configured as a hard disk drive (HDD), a solid state drive (SSD), etc. Note that only one processor may be provided, or multiple processors may be provided.

[0018] The input unit 13 is typically a keyboard or a mouse, but is not limited to these and may be any configuration that allows various data to be input. For example, the input unit 13 may be a microphone, a touch panel, etc. The output unit 14 is typically a display, but is not limited to these and may be any configuration that allows various data to be output. For example, the output unit 14 may be a speaker, a warning light, etc.

[0019] Details of each function of the control unit 10 (image conversion unit 101, cumulative curve generation unit 102, smoothing curve generation unit 103, difference curve generation unit 104, candidate detection unit 105, classification unit 106, estimation unit 107, image reconstruction unit 108, character recognition unit 109, and output control unit 110) will be described later, but will be briefly described here.

[0020] The image conversion unit 101 converts the image of the circular character string engraved on the ophthalmic lens 3 into a polar coordinate system to generate a rectangular image. The image conversion unit 101 outputs the generated rectangular image to the integral curve generation unit 102.

[0021] The integral curve generating unit 102 generates a brightness integral curve by integrating brightness values ​​for the rectangular image generated by the image converting unit 101. The integral curve generating unit 102 outputs the generated brightness integral curve to the smoothing curve generating unit 103 and the difference curve generating unit 104.

[0022] The smoothing curve generating unit 103 generates a smoothing curve by smoothing the brightness integral curve generated by the integral curve generating unit 102. The smoothing curve generating unit 103 outputs the generated smoothing curve to the difference curve generating unit 104.

[0023] The difference curve generation unit 104 calculates a difference value between the brightness integral curve generated by the integral curve generation unit 102 and the smoothed curve generated by the smoothed curve generation unit 103. The difference curve generation unit 104 generates a difference curve using the calculated difference value. The difference curve generation unit 104 outputs the generated difference curve to the candidate detection unit 105.

[0024] The candidate detection unit 105 detects, as a candidate range in which a character string appears, each range in which the absolute value of the difference value shown on the difference curve generated by the difference curve generation unit 104 is equal to or greater than a predetermined threshold value. The candidate detection unit 105 outputs the detection result to the classification unit 106.

[0025] The classification unit 106 performs clustering using the detection result by the candidate detection unit 105, and classifies the candidates in the range in which the character string appears into one or more clusters. The classification unit 106 outputs the clustering processing result to the estimation unit 107.

[0026] The estimation unit 107 estimates the range in which the character string appears using the detection result by the classification unit 106. The estimation unit 107 outputs the estimation result to the image reconstruction unit .

[0027] The image reconstruction unit 108 extracts character strings from the image within an extraction range obtained by adding a predetermined correction range to the ends of the range estimated by the estimation unit 107, and combines the extracted character strings in the order of their arrangement. The image reconstruction unit 108 outputs the processing result to the character recognition unit 109.

[0028] The character recognition unit 109 uses a predetermined character recognition method (for example, OCR (Optical Character Recognition)) to recognize the character string extracted by the image reconstruction unit 108. The character recognition unit 109 outputs the recognition result to the output control unit 110.

[0029] The output control unit 110 outputs the recognition result obtained by the character recognition unit 109 to the output unit 14. Note that the output destination of the recognition result is not limited to the output unit 14. For example, the output control unit 110 may output the recognition result to an external device.

[0030] (Image Conversion Processing) An example of the image conversion processing executed by the image conversion unit 101 will be described with reference to FIGS.

[0031] FIG. 3 shows an image 111 of the ophthalmic lens 3 captured by the camera 2. The image 111 shows a character string 31 engraved on the ophthalmic lens 3. Here, the character string 31 is made up of numbers and symbols, "44.0 / -4.50 / 106." The numbers indicate the performance of the ophthalmic lens 3, etc. An example of the engraving is laser engraving. The estimation target in this embodiment is the range in which the character string 31, "44.0 / -4.50 / 106," is captured.

[0032] The image conversion unit 101 acquires the image 111 from the camera 2. There are no particular limitations on the method of acquiring the image 111. The image conversion unit 101 may acquire the image 111 from the camera 2 using wireless communication via the communication unit 12, or may acquire the image 111 from the camera 2 using a wired connection (for example, a cable).

[0033] The image conversion unit 101 performs edge detection processing on the acquired image 111. The result of the edge detection processing is shown in Fig. 4. By the edge detection processing, a circular shape that is the outline of the ophthalmic lens 3 is detected, and the outline of the ophthalmic lens 3 is recognized in the image 111 as shown in Fig. 5.

[0034] After the edge detection process, the image conversion unit 101 converts the circular image 111 into a strip-shaped rectangular image using polar coordinate conversion.

[0035] Fig. 6 shows an image 113 that has been transformed into a strip shape by polar coordinate transformation. In Fig. 6, the horizontal direction is the x-axis and the vertical direction is the y-axis. The values ​​on the x-axis indicate the positions of character strings 31A and 31B, which will be described later. The values ​​on the y-axis indicate normalized values. This also applies to the subsequent drawings.

[0036] 6 shows an example in which a series of character strings 31 (see FIG. 3) is divided into left and right edges in an image 113 that has been converted into a strip shape by polar coordinate conversion. However, the series of character strings 31 is not necessarily divided into left and right edges. A case in which the series of character strings 31 is not divided into left and right edges will be described in a modified example below.

[0037] The character string "44.0 / -4" at the right end of image 113 is referred to as character string 31A. The character string ".50 / 106" at the left end of image 113 is referred to as character string 31B. In other words, in Figure 6, the series of character strings 31 is divided into character string 31A and character string 31B.

[0038] Although a human can visually recognize the character strings 31A and 31B in the image 113 shown in Fig. 6, it is difficult for a computer to recognize the character strings 31A and 31B in the state shown in Fig. 6. This point will be explained below.

[0039] When trying to have a computer recognize the character strings 31A and 31B in the state shown in Figure 6, one possible approach would be to search for differences in shading in the x-axis direction using well-known binarization, as described above, and find blank spaces to determine whether or not characters are present. However, in the ophthalmic lens 3, which is a contact lens for corneal correction, laser engraving is performed on transparent resin, and the curved shape makes it easy for light to reflect, making it difficult to create a contrast between the character and the background. Furthermore, because the background itself has a gradation, the character information is absorbed by the gradation of the background, making it difficult to correctly extract the character strings 31A and 31B. This inevitably makes it difficult to correctly recognize the character strings 31A and 31B.

[0040] As a method other than binarization, a method using well-known luminance integration processing can also be considered. In luminance integration processing, the luminance of each pixel is integrated in the y-axis direction and scanned in the x-axis direction. The integrated value is normalized between 0 and 1 to generate an integrated profile. FIG. 7 shows a luminance integration curve 50, which is a plot of the generated integrated profile. With regard to the presence or absence of text, ideally, the integrated value would be 0 in areas where text is present, and 1 in areas where no text is present. However, as can be seen from FIG. 7, the luminance integration curve 50 is not ideal.

[0041] Here, for example, it is conceivable to set the threshold to 0.5 and have the computer determine that parts below 0.5 contain characters. However, there is a risk that the computer may determine that there are no characters at both ends of a character, such as the beginning of the "4" in character string 31A, even though there are characters. Furthermore, the value "0.5" needs to be adjusted each time depending on the lighting conditions, which is time-consuming. For these reasons, it can be said that it is difficult to have a computer recognize character strings 31A and 31B using a method that only uses well-known luminance integration processing.

[0042] Therefore, the inventors assumed that the background gradation is contained throughout the entire original image 111, and considered the background gradation to vary smoothly within the integration profile. The inventors smoothed the integration profile using a computer, and extracted the background gradation using the difference between the smoothed curve 51 (see FIG. 8 ) and the luminance integration curve 50. Although the difference between the smoothed curve 51 and the luminance integration curve 50 is generally treated as "noise," the inventors speculated that the value of this difference could be information indicating the character position. This point will be explained below.

[0043] 8 is a diagram illustrating a luminance accumulation curve 50, a smoothing curve 51, and a difference curve 52. The luminance accumulation curve 50 is the same as that shown in FIG.

[0044] The smoothed curve 51 is generated by performing a predetermined smoothing process on the luminance integration curve 50. The smoothing process is not particularly limited, but for example, the Savitzky-Golay method may be used. The Savitzky-Golay method is a method used to differentiate and smooth digital data. The Savitzky-Golay method has the advantages of not blunting peaks despite the smoothing process, and is fast because it is essentially a weighted moving average process.

[0045] The difference curve 52 is a curve plotting the difference between the brightness integration curve 50 and the smoothing curve 51. As can be seen from the difference curve 52, the central portion where there are no characters (the portion other than the character strings 31A and 31B) is a flat line whose value fluctuates at approximately 0. Also, as can be seen from the difference curve 52, an inflection point exists in the portion where the character strings 31A and 31B are located. Therefore, the estimation unit 107 may estimate the range where the inflection point exists as the range where the character strings 31A and 31B are captured. However, because the difference curve 52 may contain noise, it is preferable to remove the noise from the difference curve 52 before making the estimation.

[0046] FIG. 9 is an enlarged view of the region 60 indicated by the dashed line in FIG. 8. Region 61 in FIG. 9 shows a portion of the central portion where no characters are present. As shown in region 61, although the value of difference curve 52 remains approximately at 0, slight irregularities are visible. Furthermore, irregularities are visible in blank portions of character string 31A where no characters are present, for example, the blank portion between "." and "0." These irregularities are noise, and are therefore preferably removed.

[0047] Fig. 10 is a diagram illustrating an example of a method for removing noise (unevenness) from the difference curve 52. In the method shown in Fig. 10, noise is removed using a threshold. As shown in Fig. 10, the candidate detection unit 105 obtains the maximum and minimum values ​​in the difference curve 52. The candidate detection unit 105 sets two thresholds based on the maximum and minimum values ​​in the difference curve 52. One threshold is one-third of the maximum value, and the other is one-third of the minimum value.

[0048] The candidate detection unit 105 quantizes the absolute value of the difference curve 52 by assigning a value of "0" if the absolute value is below a threshold value and a value of "1" if the absolute value is equal to or greater than the threshold value. The "absolute value" here refers to the distance from the zero point of the difference curve 52. Furthermore, "0" here indicates a portion where it is estimated that there is no character, and "1" indicates a portion where it is estimated that there is a character.

[0049] Fig. 11 shows the results of the quantization of "0" and "1" arranged dot by dot. In Fig. 11, the range indicated by "1" is the range detected by the candidate detection unit 105 as a candidate range where characters are to appear. Note that in Fig. 11, the "0" dots are slightly enlarged to make them easier to visually identify.

[0050] The candidate detection unit 105 recognizes position data quantized based on the presence or absence of characters as character blocks when there is a certain degree of continuity, and detects boundaries between character blocks, i.e., inflection points.

[0051] 11, the range from positions 70A to 71A indicates the range recognized as a character block. Similarly, the range from positions 72A to 73A indicates the range recognized as a character block. Positions 70A and 72A are positions where the value changes from 0 to 1, and positions 71A and 73A are positions where the value changes from 1 to 0. In other words, positions 70A to 73A can be said to be positions that indicate inflection points.

[0052] A diagram in which the positions where 0 switches to 1 are inflection points on the positive side and the positions where 1 switches to 0 are inflection points on the negative side is shown in Fig. 12. The positions of inflection points 70B to 73B shown in Fig. 12 correspond to positions 70A to 73A in Fig. 11. A diagram in which such inflection points are plotted is shown in Fig. 13.

[0053] As can be seen from FIG. 13 , a collection of vertical lines formed by plotting the inflection points overlaps the character strings 31A and 31B. Therefore, the estimation unit 107 may estimate the range in which the inflection points exist as the range in which the character strings 31A and 31B appear. Note that reference numeral 53 in FIG. 13 indicates a normalized value of the number of inflection points in the image 113. The normalized value of the number of inflection points is a value obtained by integrating the number of inflection points from position 0 on the x-axis to position 5000. Note that, as can be seen from FIG. 13 , the slash symbol " / " and the minus symbol "-" tend to be difficult to detect with the ophthalmic lens 3, which is an orthodontic contact lens. Furthermore, the last character "6" in the character string 31B (which also corresponds to the last character in the character string 31) also tends to be difficult to detect with the ophthalmic lens 3, which is an orthodontic contact lens. However, in this embodiment, the position of the character string is estimated using statistical processing rather than estimating the boundaries of individual characters. Therefore, the range in which the character string appears can be accurately estimated even when such difficult-to-detect symbols or characters are included.

[0054] The classification unit 106 performs clustering using the data of the inflection points shown in FIG. 13. In this embodiment, the x-means method is used as an example of a clustering algorithm. The x-means method is an algorithm that determines the optimal number of clusters by repeating the k-means method and using a cluster division criterion based on BIC (Bayesian information criterion). The x-means method can automatically find the optimal number of clusters, so the user does not need to set the number of clusters in advance.

[0055] 14 shows the results of the clustering analysis. In this embodiment, the inflection points are classified into two classes (clusters 80 and 81). Cluster 80 indicates a cluster of inflection points in the portion where character string 31A appears, and cluster 81 indicates a cluster of inflection points in the portion where character string 31B appears.

[0056] The classification unit 106 sets the confidence interval to 95% for each cluster 80, 81, calculates the positions of both ends (2.5% on each side), and generates a position list indicating the left and right end positions. As a result of this process, for example, for cluster 80, a position list is obtained in which the left end is 4217 and the right end is 4985. For cluster 81, a position list is obtained in which the left end is 25 and the right end is 659. Note that although the confidence interval has been described as 95% here, this utilizes the fact that the significance level in general statistical processing is 0.05 (5%), and is not limited to 95%.

[0057] Here, the width of the blank portion where no characters are present is known. Specifically, as shown in FIG. 3 , in the character string 31 "44.0 / -4.50 / 106," the length from the "4" at the left end along the contour in a counterclockwise direction to the "6" at the right end is known. Therefore, the classification unit 106 uses the position list described above to obtain the positions of the cluster ends where there is a gap the width of the blank portion where no characters are present. As a result, the classification unit 106 determines that the start position of character string 31A is 4217 and the end position of character string 31A is 4985. The classification unit 106 also determines that the start position of character string 31B is 25 and the end position of character string 31B is 659. The classification unit 106 outputs the determination results to the estimation unit 107. This enables the estimation unit 107 to estimate the range in which character strings 31A and 31B are captured.

[0058] Through this estimation, the estimation unit 107 can extract images 114A and 114B shown in (1) of FIG. 15 from the image 113 (see FIG. 6 ). The images 114A and 114B shown in (1) of FIG. 15 are images extracted from the image 113, based on the position list obtained by clustering, to extract an area estimated to contain the character strings 31A and 31B. More specifically, the image 114A is an image extracted from an area estimated to contain the character string 31A, and the image 114B is an image extracted from an area estimated to contain the character string 31B. Note that the position 811 shown in (1) of FIG. 15 indicates 25, which is the beginning position of the character string 31B. The position 812 indicates 659, which is the end position of the character string 31B. The position 801 indicates 4217, which is the beginning position of the character string 31A. The position 802 indicates 4985, which is the end position of the character string 31A. It should be noted that the x-axis values ​​shown in FIG. 15 may not match the x-axis values ​​shown in FIG.

[0059] As can be seen from image 114B shown in (1) of Figure 15, the "6" is cut off at the end (right end) of character string 31B. In other words, the "6" does not fit into the cut-out image 114B. Therefore, there are cases where cutting out using the position list described above is not sufficient. Needless to say, it should be noted that Figure 15 shows an example of the present invention, and characters at the end are not always cut off.

[0060] Because the length of character string 31 (the sum of the lengths of character string 31A and character string 31B) is known, the image reconstruction unit 108 performs processing to correct the shortfall by taking the length of character string 31 into consideration. As an example of a specific correction method, the image reconstruction unit 108 adds a correction range 91 of a predetermined width extending from position 812 to the right, as shown in FIG. 15 (2). Similarly, the image reconstruction unit 108 adds a correction range 90 of a predetermined width extending from position 801 to the left, as shown in FIG. 15 (2). This correction processing generates images 115A and 115B with corrected widths, as shown in FIG. 15 (3). It can be seen from the newly cropped image 115B that "6" is included.

[0061] Next, the image reconstructing unit 108 swaps the newly extracted images 115A and 115B and combines them in the order of the character strings. By this combination, an image 116 is generated, as shown in (4) of FIG.

[0062] Next, the image reconstruction unit 108 deletes the slashes that are part of the character string, as shown in (5) of FIG. 15. Because the positions of the slashes are known, the image reconstruction unit 108 can delete the slashes by using known information. By deleting the slashes, the image 116 is divided into images 116A, 116B, and 116C. Note that this deletion process does not have to be performed. Therefore, the morphological transformation and OCR described below may be performed on the images 116A, 116B, and 116C, or on the image 116.

[0063] 16 is a diagram illustrating an example of a method for enhancing contrast. In this embodiment, morphological transformation is used for contrast enhancement. Morphological transformation is a method for enhancing contrast by performing dilation and contraction operations to emphasize the boundaries and structures of an object.

[0064] (1) in Fig. 16 shows the state before morphological transformation. (2) in Fig. 16 shows the result of processing to emphasize the vertical component. (3) in Fig. 16 shows the result of processing to emphasize the horizontal component. (4) in Fig. 16 shows the result of processing to emphasize the right diagonal component. (5) in Fig. 16 shows the result of processing to emphasize the left diagonal component. (6) in Fig. 16 shows the result of processing all of (2) to (5). In general, the result of (6) is expected to be the most desirable, but it is not necessary to use the result of (6). It is sufficient to repeatedly perform experiments and simulations and select the processing from (2) to (6) that produces the best recognition results, as described below.

[0065] The character recognition unit 109 uses a character recognition method such as OCR on the morphologically transformed image to recognize characters from the images 116A, 116B, and 116C, thereby obtaining the recognition result "44.00-4.50106".

[0066] (Processing Flow) Next, the flow of processing executed by the information processing device 1 will be described with reference to Fig. 17. Fig. 17 is a flowchart showing an example of processing executed by the information processing device 1.

[0067] In step S1, the image conversion unit 101 acquires an image 111 from the camera 2 (see FIG. 3). The image conversion unit 101 performs edge detection processing on the acquired image 111 (see FIG. 4). The edge detection processing detects a circular shape that is the outline of the ophthalmic lens 3, and the outline of the ophthalmic lens 3 is recognized in the image 111 (see FIG. 5).

[0068] The process proceeds to step S2, where the image conversion unit 101 converts the circular image 111 into a strip-shaped image 113 using polar coordinate conversion (see FIG. 6). By the process of step S2, the series of characters 31 is divided into characters 31A and 31B at both ends of the image 113.

[0069] The process proceeds to step S3, where the integral curve generator 102 integrates the luminance values ​​of pixels arranged in the y-axis direction, perpendicular to the x-axis direction, along which the character strings 31A and 31B are arranged. The integral curve generator 102 normalizes the integrated values ​​between 0 and 1 to generate an integrated profile. The integral curve generator 102 plots the integrated profile to generate a luminance integral curve 50 (see FIG. 8 ).

[0070] The process proceeds to step S4, where the smoothed curve generating unit 103 smooths the luminance integral curve 50 generated in the process of step S3 using the Savitzky-Golay method or the like to generate a smoothed curve 51 (see FIG. 8).

[0071] The process proceeds to step S5, where the difference curve generating unit 104 calculates the difference between the luminance integration curve 50 and the smoothed curve 51, and plots the difference to generate a difference curve 52 (see FIG. 8).

[0072] The process proceeds to step S6, where the candidate detection unit 105 quantizes the difference curve 52 generated in step S5 by assigning a value of "0" if the absolute value is below a threshold and a value of "1" if the absolute value is equal to or greater than the threshold (see FIG. 11). The candidate detection unit 105 recognizes, from the position data quantized based on the presence or absence of characters, those that have a certain degree of continuity as character blocks, and detects the boundaries of the character blocks, i.e., inflection points (see FIG. 12).

[0073] The candidate detection unit 105 may detect each range in the difference curve 52 where the absolute value of the difference value shown on the difference curve 52 is equal to or greater than a predetermined threshold as a candidate range in which the character strings 31A and 31B appear. Alternatively, the candidate detection unit 105 may detect each range in which an inflection point exists as a candidate range in which the character strings 31A and 31B appear (see FIG. 13 ).

[0074] The process proceeds to step S7, where the classification unit 106 classifies the multiple candidates detected in step S6 into one or more clusters based on their positions in the x-axis direction using the x-means algorithm or the like (see FIG. 14 ). For each cluster 80, 81, the classification unit 106 calculates the positions of both ends of the cluster, setting the confidence interval at 95%, and generates a position list indicating the positions of the left and right ends. The classification unit 106 uses the position list to obtain the positions of the cluster ends, which are separated by a gap the width of a blank space where no characters are present. The classification unit 106 determines position 4217 as the beginning position of character string 31A and position 4985 as the end position of character string 31A. The classification unit 106 also determines position 25 as the beginning position of character string 31B and position 659 as the end position of character string 31B.

[0075] The process proceeds to step S8, where the estimation unit 107 estimates, based on the determination result of step S7, that the range from positions 4217 to 4985 is the range in which character string 31A appears. The estimation unit 107 also estimates that the range from positions 25 to 659 is the range in which character string 31B appears.

[0076] The process proceeds to step S9, where the estimation unit 107 uses the estimation result of step S8 to cut out images 114A and 114B from the image 113 generated in the process of step S2 (see (1) in FIG. 15).

[0077] The process proceeds to step S10, where the image reconstructing unit 108 adds correction ranges 90 and 91 of a predetermined width to the images 114A and 114B extracted in step S9 (see (2) in FIG. 15 ). The image reconstructing unit 108 extracts new images 115A and 115B by adding the correction ranges 90 and 91 (see (3) in FIG. 15 ). As a result, the newly extracted image 115B can contain the character "6." The image reconstructing unit 108 swaps the left and right sides of the newly extracted images 115A and 115B and combines them in the order of the character string to generate image 116 (see (4) in FIG. 15 ). The image reconstructing unit 108 then deletes the slash from image 116 to generate images 116A, 116B, and 116C (see (5) in FIG. 15 ).

[0078] The process proceeds to step S11, where the character recognition unit 109 performs morphological transformation on the images 116A, 116B, and 116C generated in step S10 (see FIG. 16). The character recognition unit 109 uses a character recognition method such as OCR on the morphologically transformed images to recognize characters from the images 116A, 116B, and 116C.

[0079] The process proceeds to step S12, where the output control unit 110 outputs the recognition result obtained in step S11 to the output unit 14.

[0080] As described above, the estimation method according to this embodiment is a method for estimating a character string range executed by one or more information processing devices 1, and includes: a difference curve generation process for generating a difference curve 52 indicating the difference between a brightness accumulation curve 50 generated by accumulating the brightness values ​​of pixels arranged in a second direction perpendicular to a first direction in which the character string 31 is arranged, along a first direction in which the character string 31 is arranged, and a smoothed curve 51 obtained by smoothing the brightness accumulation curve 50, for an image 111 capturing the character string 31 on a target object; and an estimation process for estimating the range in which the character string 31 appears in the image 111 based on the difference curve 52. Note that the processing flow in the flowchart shown in FIG. 17 is an example, and steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0081] (Operational Effects) As described above, the information processing device 1 according to the embodiment provides the following operational effects.

[0082] The information processing device 1 includes: a brightness accumulation curve 50 generated by accumulating the brightness values ​​of each pixel arranged in a second direction (y-axis direction) perpendicular to a first direction (x-axis direction) in which the character string 31 is arranged for an image 111 captured of a character string 31 on a target object (ophthalmic lens 3); a difference curve generation unit 104 that generates a difference curve 52 showing the difference between the brightness accumulation curve 50 and a smoothed curve 51 obtained by smoothing the brightness accumulation curve 50; and an estimation unit 107 that estimates the range in which the character string 31 appears in the image 111 based on the difference curve 52.

[0083] As a result of experiments conducted by the inventors, it was found that even in a low-contrast image 111, the difference curve 52 indicating the difference between the brightness accumulation curve 50 and the smoothing curve 51 shows a relatively clear boundary between the range in which the character string 31 appears and the range in which the character string 31 does not appear. Therefore, according to the above configuration, the difference curve 52 indicating the difference between the brightness accumulation curve 50 and the smoothing curve 51 is generated, and the range in which the character string 31 appears in the image 111 is estimated based on the generated difference curve 52. As described above, since the difference curve 52 shows a relatively clear boundary between the range in which the character string 31 appears and the range in which the character string 31 does not appear, according to this configuration, it is possible to accurately estimate the range in which the character string 31 appears even in a low-contrast image 111.

[0084] The information processing device 1 may also include a candidate detection unit 105 that detects, on the difference curve 52, each range in which the absolute value of the difference value indicated on the difference curve 52 is equal to or greater than a predetermined threshold, as a candidate range in which the character string 31 appears. The estimation unit 107 may estimate, based on the detected candidates, the range in the image 111 in which the character string 31 appears.

[0085] As a result of experiments conducted by the inventors, it was found that the range in which the absolute value of the difference value shown on the difference curve 52 is relatively large is often included in the range in which the character string 31 appears. For this reason, as described above, it is preferable to employ a configuration in which each range in which the absolute value of the difference value shown on the difference curve 52 is equal to or greater than a predetermined threshold is detected as a candidate range in which the character string 31 appears, and the range in the image 111 in which the character string 31 appears is estimated based on the detected candidates. This makes it possible to further improve the accuracy of estimating the range in which the character string 31 appears.

[0086] The information processing device 1 may also include a classification unit 106 that classifies the multiple candidates detected by the candidate detection unit 105 into one or more clusters based on the positions of the candidates in the first direction. The estimation unit 107 may estimate that a range defined by the multiple candidates classified into one cluster at a predetermined confidence level is a range in which the series of character strings 31 appears.

[0087] Experiments conducted by the inventors have revealed that the value of the difference curve 52 fluctuates periodically within the range in which the series of character strings 31 is captured. Therefore, within the range in which the series of character strings 31 is captured, the range in which the absolute value of the difference value is equal to or greater than a predetermined threshold is detected discretely along the x-axis direction. However, it has been found that there is a pattern to how candidates are detected, and within the range in which the series of character strings 31 is captured, each candidate is detected in a relatively close-up position. Therefore, as described above, it is preferable to classify the detected candidates into one or more clusters based on their positions in the x-axis direction, and estimate the range defined by the candidates classified into one cluster with a predetermined confidence level as the range in which the series of character strings 31 is captured. This makes it possible to accurately estimate the range in which the series of character strings 31 is captured from the discrete detection results of the candidates.

[0088] The information processing device 1 may also include an image conversion unit 101 that converts an area in the image 111 (original image) in which character strings 31 are arranged in a circular pattern into polar coordinates to generate an image 113 (see Figure 6) in which the character strings 31 are arranged in a first direction.

[0089] According to the above configuration, it is possible to automatically estimate the range in which the character strings 31 appear in the image 111 in which the character strings 31 are arranged in a circular pattern.

[0090] Furthermore, when the image 113 generated by the image conversion unit 101 shows a circularly arranged character string 31 split at both ends of the image 113, the information processing device 1 may be provided with an image reconstruction unit 108 that cuts out from the image 113, among the ranges estimated by the estimation unit 107, ranges corresponding to the character strings 31A and 31B split at both ends of the image 113, and joins them in the order of the arrangement of the character strings 31A and 31B.

[0091] When a region in which character strings 31 are arranged in a circular pattern is subjected to polar coordinate transformation, the character strings 31 may be divided. Therefore, with the above configuration, when such division occurs, the ranges corresponding to the divided character strings 31A and 31B are cut out from the image 113 and joined together in the order of arrangement of the character strings 31. This makes it possible to perform character recognition in accordance with the original order of arrangement of the character strings 31, even when the character strings 31 are divided and appear at both ends of the image 113.

[0092] The image reconstruction unit 108 may cut out the character strings 31A and 31B from the image 113 within a cutout range obtained by adding predetermined correction ranges 90 and 91 to the ends (positions 801 and 812) of the range estimated by the estimation unit 107.

[0093] As described above, according to this embodiment, it is possible to accurately estimate the range in which the character string 31 appears even in a low-contrast image 111. However, it is possible that some characters do not fall within the estimated range due to factors such as the shape of the characters and how they appear. Therefore, as described above, the character strings 31A and 31B may be cut out from the image 113 using a cutout range that adds predetermined correction ranges 90 and 91 to the ends of the estimated range. In this way, even if some characters do not fall within the estimated range, the range in which the some characters appear can be filled in with the correction range, making it possible to perform character recognition on a complete string of character strings 31.

[0094] The information processing device 1 may also include a character recognition unit 109 that performs character recognition on the range estimated by the estimation unit 107 in the image 113, and an output control unit 110 that outputs the results of the character recognition by the character recognition unit 109 to a specified output device.

[0095] According to the above configuration, it is possible to accurately estimate the range in which the character string 31 appears even in an image 111 with low contrast, and it is possible to automatically recognize the character string 31 in that range and present the results of the character recognition to the user via an output device.

[0096] 18 to 19, a modified example will be described. In the above embodiment, the case where the character string 31 is divided into left and right parts has been described, but in the modified example, the case where the character string 31 is not divided into left and right parts but remains continuous will be described.

[0097] Fig. 18 shows a group of vertical lines with inflection points plotted for a series of character strings 31. In Fig. 18, as compared with Fig. 13, there are no relatively wide blank spaces with no characters between the groups of vertical lines, and therefore the estimation unit 107 may estimate that the series of character strings 31 are captured within the range of the group of vertical lines.

[0098] Fig. 19 shows the results of clustering analysis of the graph shown in Fig. 18. In the example shown in Fig. 19, the data is classified into five classes (clusters 82 to 86). The classification unit 106 sets the confidence interval for each of the clusters 82 to 86 at 95%, calculates the positions of both ends of the interval, and generates a position list indicating the left and right end positions.

[0099] The classification unit 106 may determine the beginning (left end) of the cluster 82 as the beginning position of the character string 31. The classification unit 106 may also determine the end (right end) of the cluster 86 as the end position of the character string 31. The estimation unit 107 may estimate that the range from the beginning position of the cluster 82 to the end position of the cluster 86 is the range in which the character string 31 appears.

[0100] If the portion from the beginning of cluster 82 to the end of cluster 86 is extracted, the "6" at the end of character string 31 will be cut off, as in the above-described embodiment. Therefore, as in the above-described embodiment, the image reconstruction unit 108 may add a correction range of a predetermined width to the beginning of cluster 82 and the end of cluster 86. This makes it possible to extract the "6" in the same way as in the above-described embodiment.

[0101] In this way, the present invention is applicable not only to cases where the character string 31 is divided into left and right parts, but also to cases where it is not divided into left and right parts.

[0102] Other Embodiments The present invention is not limited to medical devices such as the ophthalmic lens 3, and can be applied to various objects on which any character string is engraved. For example, the present invention can be applied to recognizing a character string engraved on a casting or a tablet.

[0103] [Example of implementation by software] The functions of the information processing device 1 are realized by a program (estimation program) for causing a computer to function as the information processing device 1, and a program for causing a computer to function as each control block of the information processing device 1.

[0104] In this case, the information processing device 1 includes a computer having at least one device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The computer executes the program to realize each of the functions described in the above embodiment.

[0105] The program may be recorded on one or more computer-readable recording media, rather than being temporary. The recording media may or may not be included in the information processing device 1. In the latter case, the program may be supplied to the information processing device 1 via any wired or wireless transmission medium.

[0106] In addition, some or all of the functions of each control block can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each control block is formed is also included in the scope of the present invention. In addition, the functions of each control block can also be realized by, for example, a quantum computer.

[0107] Furthermore, each process described in the above embodiment may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the information processing device 1 or on another device (for example, an edge computer or a cloud server).

[0108] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims.

[0109] REFERENCE SIGNS LIST 100 Information processing system 1 Information processing device 101 Image conversion unit 102 Integration curve generation unit 103 Smoothing curve generation unit 104 Difference curve generation unit 105 Candidate detection unit 106 Classification unit 107 Estimation unit 108 Image reconstruction unit 109 Character recognition unit 110 Output control unit 50 Luminance integration curve 51 Smoothing curve 52 Difference curve

Claims

1. An information processing device comprising: a difference curve generation unit that generates a difference curve showing the difference between a brightness accumulation curve generated by accumulating the brightness values ​​of each pixel arranged in a second direction perpendicular to a first direction in which the character string is arranged, and a smoothed curve obtained by smoothing the brightness accumulation curve, for an image of the character string on a target object; and an estimation unit that estimates the range in which the character string appears in the image based on the difference curve.

2. An information processing device as described in claim 1, further comprising a candidate detection unit that detects each range in the difference curve where the absolute value of the difference value shown on the difference curve is equal to or greater than a predetermined threshold as a candidate range in which the character string appears, and the estimation unit estimates the range in the image in which the character string appears based on the detected candidates.

3. An information processing device as described in claim 2, further comprising a classification unit that classifies multiple candidates detected by the candidate detection unit into one or more clusters based on the positions of the candidates in the first direction, and wherein the estimation unit estimates that the range defined by the multiple candidates classified into one cluster at a predetermined confidence level is the range in which the series of character strings appears.

4. An information processing device according to any one of claims 1 to 3, comprising an image conversion unit that generates the image in which the character strings are arranged in the first direction by polar coordinate conversion of an area in the original image in which the character strings are arranged in a circular pattern.

5. An information processing device as described in claim 4, further comprising an image reconstruction unit that, when the circularly arranged character string is shown split at both ends of the image in the image generated by the image conversion unit, cuts out each range from the image estimated by the estimation unit that corresponds to the character string that is shown split at both ends of the image and joins them in the order of the character string arrangement.

6. The information processing device according to claim 5, wherein the image reconstruction unit cuts out the character string from the image within a cutout range obtained by adding a predetermined correction range to the ends of the range estimated by the estimation unit.

7. An information processing device according to any one of claims 1 to 3, comprising: a character recognition unit that performs character recognition on the range estimated by the estimation unit in the image; and an output control unit that outputs the results of character recognition by the character recognition unit to a predetermined output device.

8. A character string range estimation method executed by one or more information processing devices, comprising: a difference curve generation process for generating a difference curve showing the difference between a brightness accumulation curve generated by accumulating the brightness values ​​of each pixel arranged in a second direction perpendicular to a first direction in which the character string is arranged, and a smoothed curve obtained by smoothing the brightness accumulation curve, for an image capturing the character string on a target object; and an estimation process for estimating the range in which the character string appears in the image based on the difference curve.

9. A character string range estimation program for causing a computer to function as the information processing device according to claim 1, the character string range estimation program causing a computer to function as the difference curve generating section and the estimation section.

Images