Character reading system, character reading method, and program

Abstract

To make it possible to read both engraved characters and printed characters.SOLUTION: A character reading system acquires a target image that is an image of an outdoor metal name plate installed on a transformer, the image of the metal name plate on which engraved characters and printed characters are described, performs image processing on the target image, extracts a feature amount from the target image applied with the image processing, selects master information according to a pattern of the target image based on the feature amount from master information provided for every pattern of the metal name plate, the master information in which the feature amount extracted from the image obtained by applying the image processing to a master image being the image of the metal name plate and information on the arrangement of the engraved characters and the printed characters are linked to each other, and based on information on the arrangement of characters indicated by the selected master information, performs different types of character recognition processing on the engraved characters and the printed characters in the target image applied with the image processing to read the characters indicated in the target image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a character reading system, a character reading method, and a program. [Background technology]

[0002] As one measure to prevent characters from becoming illegible due to deterioration over time, stamped characters are used in which characters are stamped into metal or the like and are indicated by indentations in the metal or the like (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 7-149267 Summary of the Invention [Problem to be solved by the invention]

[0004] Metal nameplates for transformers installed on utility poles often use both embossed characters stamped on the metal nameplate and printed characters that are written by applying ink to the metal nameplate. To deal with this, it is preferable to be able to read both the embossed characters and the printed characters.

[0005] An object of the present invention is to provide a character reading system, a character reading method, and a program that can solve the above-mentioned problems. [Means for solving the problem]

[0006] According to a first aspect of the present invention, a character reading system includes a target image acquisition unit that acquires a target image, which is an image of an outdoor metal nameplate installed on a transformer, the target image being an image of the metal nameplate on which embossed characters and printed characters are written; an image processing unit that performs image processing on the target image; a feature extraction unit that extracts features from the target image to which the image processing has been applied; a master information selection unit that selects master information corresponding to the pattern of the target image based on the features from master information provided for each pattern of the metal nameplate, the master information being linked to features extracted from an image to which the image processing has been applied to the master image, which is the image of the metal nameplate, and arrangement information of the embossed characters and the printed characters; and a character reading unit that performs different character recognition processing for the embossed characters and the printed characters on the target image to which the image processing has been applied, based on the character arrangement information indicated by the selected master information, to read the characters shown in the target image.

[0007] According to a second aspect of the present invention, a character reading system includes an imaging device, an image processing device, an image recognition device, and a character reading device. The imaging device captures a target image, which is an image of an outdoor metal nameplate installed on a transformer, the target image being an image of the metal nameplate on which embossed characters and printed characters are written. The image processing device performs image processing on the target image. The image recognition device includes a feature extraction unit that extracts features from the target image to which the image processing has been applied, and a master information selection unit that selects master information corresponding to the pattern of the target image based on the features from master information provided for each pattern of the metal nameplate, the master information being linked to features extracted from an image to which the image processing has been applied to the master image, which is the image of the metal nameplate, and arrangement information of the embossed characters and the printed characters. The character reading device performs different character recognition processing for the embossed characters and the printed characters on the target image to which the image processing has been applied, based on the character arrangement information indicated by the selected master information, to read the characters shown in the target image.

[0008] According to a third aspect of the present invention, a character reading method includes the steps of: acquiring a target image, which is an image of an outdoor metal nameplate installed on a transformer, the target image being an image of the metal nameplate on which embossed characters and printed characters are written; performing image processing on the target image; extracting features from the target image to which the image processing has been applied; selecting master information corresponding to the pattern of the target image based on the features from among master information provided for each pattern of the metal nameplate, the master information linking the features extracted from an image to which the image processing has been applied to the master image, which is the image of the metal nameplate, and arrangement information of the embossed characters and the printed characters; and performing different character recognition processing for the embossed characters and the printed characters on the target image to which the image processing has been applied, based on the character arrangement information indicated by the selected master information, to read the characters indicated in the target image.

[0009] According to a fourth aspect of the present invention, the program causes a computer to execute the steps of: acquiring a target image, which is an image of an outdoor metal nameplate installed on a transformer, the target image being an image of the metal nameplate on which embossed characters and printed characters are written; performing image processing on the target image; extracting features from the target image to which the image processing has been applied; selecting master information corresponding to the pattern of the target image based on the features from among master information provided for each pattern of the metal nameplate, the master information linking the features extracted from an image to which the image processing has been applied to the master image, which is the image of the metal nameplate, and arrangement information of the embossed characters and the printed characters; and reading the characters shown in the target image by performing different character recognition processing for the embossed characters and the printed characters on the target image to which the image processing has been applied based on the character arrangement information indicated by the selected master information. Effect of the Invention

[0010] According to this invention, both embossed characters and printed characters can be read. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram illustrating an example of a device configuration of a character reading system according to an embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of the arrangement of characters on a metal nameplate of a transformer. [Diagram 3] FIG. 2 is a schematic block diagram showing an example of a functional configuration of an embossed character reading server device according to an embodiment. [Figure 4] 2 is a schematic block diagram illustrating an example of a functional configuration of a Web server device according to an embodiment. FIG. [Diagram 5] 2 is a schematic block diagram illustrating an example of a functional configuration of an image identification server device according to an embodiment. FIG. [Figure 6] 2 is a schematic block diagram illustrating an example of a functional configuration of a printed character reading server device according to an embodiment. FIG. [Figure 7] FIG. 2 is a schematic block diagram illustrating an example of a functional configuration of a database server device according to the embodiment. [Figure 8] 2 is a schematic block diagram illustrating an example of a functional configuration of a management terminal device according to the embodiment. [Figure 9] 11 is a diagram illustrating an example of a processing procedure in which the character reading system according to the embodiment generates and registers master information. [Figure 10] 11 is a diagram illustrating an example of a processing procedure in which the character reading system according to the embodiment selects master information corresponding to a target image. [Figure 11] 11 is a diagram showing an example of a processing procedure in which the character reading system according to the embodiment reads printed characters and embossed characters from a processed target image using master information. FIG. [Figure 12] 1 is a diagram illustrating an example of a configuration of a character reading system according to an embodiment. [Figure 13] FIG. 13 is a diagram showing another example of the configuration of the character reading system according to the embodiment. [Figure 14] 4 is a flowchart showing an example of a processing procedure in a character reading method according to the embodiment. [Figure 15] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The following describes embodiments of the present invention, but the following embodiments do not limit the scope of the invention. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. Fig. 1 is a schematic diagram showing an example of the device configuration of a character reading system according to an embodiment. In the configuration shown in Fig. 1, the character reading system 1 includes an imaging device 11, a mobile terminal device 12, a stamped character reading server device 21, a Web server device 22, an image recognition server device 23, a printed character reading server device 24, a database server device 25, and a management terminal device 31.

[0013] The character reading system 1 reads characters on the metal nameplate (metal nameplate) of a transformer installed on a utility pole. This reduces the burden on workers performing on-site maintenance and inspection of transformers to input information about the transformer when recording their work. The metal nameplate that is the target of character reading by the character reading system 1 is also referred to as the target nameplate. In particular, the character reading system 1 reads both embossed characters and printed characters from a metal plate on which both are written. Embossed characters are characters that are written on the metal plate by stamping the characters and are indicated by indentations on the metal plate. Printed characters are characters that are indicated by ink that is applied to the metal plate.

[0014] In the case of printed characters, if the ink falls off the metal plate due to aging, the characters cannot be read from the metal plate. On the other hand, in the case of stamped characters, since the characters are displayed by the deformation of the metal plate, there is less possibility that the characters will disappear due to aging than in the case of printed characters.

[0015] On the other hand, in the case of printed characters, for example, by writing characters in black or other ink with a low brightness on the surface of a bright metal plate, the contrast between the metal plate and the ink can be made strong. On the other hand, in the case of stamped characters, the characters are indicated by the shadows caused by the indentations in the metal plate, and the contrast between the characters and the background is weaker than in the case of printed characters. For this reason, stamped characters are generally more difficult to read than printed characters. For example, when a general OCR (Optical Character Recognition) is applied to a captured image of a metal plate, the recognition accuracy of stamped characters is lower than that of printed characters.

[0016] FIG. 2 is a diagram showing an example of the arrangement of characters on a metal nameplate of a transformer. On the metal nameplate of the transformer shown in Figure 2, the model, serial number, year of manufacture, and manufacturer name of the transformer are shown in areas A11, A12, A13, and A14, respectively. Of these, the serial number (area A12) is shown in stamped characters. Meanwhile, the model (area A11), year of manufacture (area A13), and manufacturer name (area A14) are all shown in printed characters.

[0017] The character reading system 1 stores in advance master information indicating the arrangement of character areas on the metal nameplate and whether each character area is stamped or printed. When an image of the transformer's metal nameplate is acquired, the character reading system 1 reads the characters using the master information. Because the ease of reading stamped characters and printed characters differs, the character reading system 1 reads stamped characters and printed characters using different methods.

[0018] If the manufacturer or year of manufacture is different, the arrangement of characters on the metal nameplate, or the distinction between stamped characters and printed characters for each character area may differ. Therefore, the character reading system 1 stores master information for each pattern of the metal nameplate in advance. The pattern of the metal nameplate here refers to the design pattern of the metal nameplate, including the arrangement pattern of the stamped characters and the printed characters. The pattern of the metal nameplate shown in the captured image of the metal nameplate is also referred to as the pattern of the captured image. The character reading system 1 selects one of the master information items according to a captured image of the target nameplate, and reads characters from the target nameplate using the selected master information item.

[0019] The master information is generated based on an image of the metal plate selected for each pattern of the metal plate. For example, an administrator of the character reading system 1 selects a captured image of a metal nameplate to be used for generating master information for each pattern of the metal nameplate. The selected image is called a master image, and the metal nameplate shown in the master image is called a master nameplate.

[0020] The character reading system 1 performs image processing on the master image to make it easier for a computer to identify the pattern on the metal nameplate. The master image to which the image processing has been applied is called a processed master image. The character reading system 1 extracts feature amounts from the processed master image. The character reading system 1 also displays the master image and accepts a user operation indicating the arrangement of each of the embossed characters and printed characters in the master image. The character reading system 1 generates arrangement information of each of the embossed characters and printed characters in the master image based on the user operation. The arrangement information of each of the embossed characters and printed characters in the master image can be regarded as the arrangement information of each of the embossed characters and printed characters on the master nameplate.

[0021] The master information includes the master image, feature amounts extracted from the processed master image, and layout information for each of the stamped characters and printed characters on the master nameplate. The master image is used for confirmation by a user such as the administrator of the character reading system 1. The feature amounts are used to select master information corresponding to the pattern of the target nameplate from among the master information for each pattern of the metal nameplate by comparing with feature amounts extracted from the image of the target nameplate. The character layout information (layout information for each of the stamped characters and printed characters) is used to identify the target area in the image of the target nameplate where character reading is to be performed.

[0022] The image capturing device 11 is carried by a worker to the site where the transformer is installed, and captures an image of the target nameplate according to the worker's operation. The site where the transformer is installed is also simply referred to as the site. As described above, the target nameplate is a metal nameplate of a transformer installed on a utility pole. Therefore, the target nameplate is an outdoor metal nameplate installed on a transformer. The captured image of the target nameplate is also referred to as a target image. The imaging device 11 may be a digital camera.

[0023] The mobile terminal device 12 is carried by a worker to the site where the transformer is installed, acquires a target image from the imaging device 11, and transmits the acquired target image to the stamp character reading server device 21. The mobile terminal device 12 is a device capable of communicating with the stamp character reading server device 21 from the site. A mobile phone such as a smartphone can be used as the mobile terminal device 12. A storage device such as a memory card may be used to transfer the target image from the imaging device 11 to the mobile terminal device 12. Alternatively, the mobile terminal device 12 may acquire the target image by performing short-range wireless communication with the imaging device 11.

[0024] The embossed character reading server device 21 is a communication target of the mobile terminal device 12 . The embossed character reading server device 21 receives a target image from the mobile terminal device 12 and performs image processing on the received target image. The embossed character reading server device 21 transmits the target image to which the image processing has been applied to the Web server device 22. The target image to which the image processing has been applied is also referred to as a processed target image. Furthermore, the embossed character reading server device 21 reads the embossed characters from the processed target image based on the master information received from the Web server device 22. The embossed character reading server device 21 transmits the read embossed characters to the mobile terminal device 12 together with the printed characters read by the printed character reading server device 24.

[0025] FIG. 3 is a schematic block diagram showing an example of the functional configuration of the embossed character reading server device 21. As shown in FIG. 3, the embossed character reading server device 21 includes a first communication unit 110, a first storage unit 210, and a first control unit 310. The first storage unit 210 includes an image processing library storage unit 211. The first control unit 310 includes an image processing unit 311, and an embossed character reading processing unit 312.

[0026] The first communication unit 110 communicates with other devices. In particular, the first communication unit 110 communicates with the mobile terminal device 12 to receive the target image as image data. The first communication unit 110 corresponds to an example of a target image acquisition unit. In addition, the first communication unit 110 transmits the reading results of the embossed characters and printed characters from the processed target image to the mobile terminal device 12. In the character reading results, for example, the read characters are shown as text data. Moreover, the first communication unit 110 transmits the processed target image as image data to the Web server device 22. Moreover, the first communication unit 110 receives the results of reading the printed characters and the master information from the Web server device 22.

[0027] The first storage unit 210 stores various data. The function of the first storage unit 210 is executed by using a storage device included in the embossed character reading server device 21. The image processing library storage unit 211 stores an image processing library. The library here is a program component provided in an executable format. The image processing library is a program component capable of executing image processing. The image processing library storage unit 211 stores an image processing library including binarization processing and edge extraction.

[0028] The first control unit 310 executes various processes by controlling each unit of the embossed character reading server device 21. The functions of the first control unit 310 are executed by a CPU (Central Processing Unit) included in the embossed character reading server device 21 reading and executing a program from the first storage unit 210.

[0029] The image processing unit 311 performs image processing on the target image to make it easier for the computer to identify the pattern on the metal nameplate. In particular, the image processing unit 311 performs binarization processing and edge extraction as image processing on the target image. The image processing unit 311 performs image processing on the target image by executing the image processing library stored in the image processing library storage unit 211. The embossed character reading server device 21 corresponds to an example of an image processing device in that the image processing unit 311 performs image processing on the target image.

[0030] The embossed character reading processor 312 reads the embossed characters from the processed target image. The master information that the embossed character reading server device 21 receives from the Web server device 22 indicates the arrangement of the embossed characters and the printed characters in the target image. The embossed character reading processor 312 reads the embossed characters from the processed target image based on the character arrangement information indicated by the master information.

[0031] As described above, embossed characters are more difficult to read than printed characters, and it is considered that high accuracy cannot be obtained by using a general OCR to read embossed characters. Therefore, the embossed character reading processor 312 reads embossed characters by pattern matching. For example, the embossed character reading processor 312 stores in advance the characters of the font used for embossed characters, and reads the embossed characters by pattern matching with the characters of the font.

[0032] The position of the embossed characters in the post-processing target image is indicated by the master information. Based on the master information, the embossed character reading processor 312 extracts an image of the area in which the embossed characters are written from the post-processing target image. The embossed character reading processor 312 then performs pattern matching on the extracted image to read the embossed characters. The position of a character here refers to the position of the area in which the character is written. The position of a character is also called the arrangement of the character. When embossed characters are written in multiple positions on the processed target image, the embossed character reading processing unit 312 reads the embossed characters from each of the multiple positions where the embossed characters are written.

[0033] Web server device 22 receives the processed target image from embossed character reading server device 21. Stamped character reading server device 21 transmits the received processed target image to image recognition server device 23, and receives master information corresponding to the processed target image from image recognition server device 23. Web server device 22 also transmits the processed target image and the master information to printed character reading server device 24, causing it to read the printed characters from the target image. Web server device 22 receives the printed character reading results from printed character reading server device 24, and transmits the received reading results to embossed character reading server device 21. Furthermore, the Web server device 22 manages the master information.

[0034] FIG. 4 is a schematic block diagram showing an example of the functional configuration of the Web server device 22. As shown in FIG. 4, the Web server device 22 includes a second communication unit 120, a second storage unit 220, and a second control unit 320. The second control unit 320 includes a query processing unit 321 and a master management unit 322.

[0035] The second communication unit 120 communicates with other devices. For example, the second communication unit 120 receives the processed target image transmitted by the embossed character reading server device 21. The second communication unit 120 also transmits the processed target image to the image recognition server device 23 and receives master information corresponding to the transmitted processed target image. The second communication unit 120 also transmits the processed target image and the master information to the printed character reading server device 24, and receives the printed character reading result from the processed target image. The second communication unit 120 transmits the printed character reading result to the embossed character reading server device 21.

[0036] The second storage unit 220 stores various data. The function of the second storage unit 220 is executed using a storage device included in the Web server device 22. The second control unit 320 executes various processes by controlling each unit of the Web server device 22. The functions of the second control unit 320 are executed by the CPU included in the Web server device 22 reading out a program from the second storage unit 220 and executing it.

[0037] The query processing unit 321 acquires master information from image recognition server device 23, and acquires the printed character reading results from printed character reading server device 24. Specifically, the query processing unit 321 transmits the processed target image transmitted from embossed character reading server device 21 to image recognition server device 23 via second communication unit 120, and acquires master information corresponding to the transmitted processed target image. The query processing unit 321 also transmits the processed target image and the master information to printed character reading server device 24 via second communication unit 120, and acquires the printed character reading results from the processed target image.

[0038] The query processor 321 also extracts the printed character area and the stamped character area indicated by the master information from the target image. The obtained images are called extracted images. The extracted images are referred to when an operator checks the results of reading the characters. The master management unit 322 manages the master information. For example, the master management unit 322 causes the management terminal device 31 to display an operation screen for generating, changing, and deleting the master information.

[0039] The image identification server device 23 selects master information corresponding to the processed target image transmitted from the Web server device 22. The master information corresponding to the processed target image is master information corresponding to the pattern of the metal nameplate shown in the processed target image. The image identification server device 23 estimates and selects the master information corresponding to the pattern of the metal nameplate shown in the processed target image by comparing the feature amount extracted from the processed target image with the feature amount shown in the master information.

[0040] FIG. 5 is a schematic block diagram showing an example of the functional configuration of the image identification server device 23. As shown in FIG. 5, the image recognition server device 23 includes a third communication unit 130, a third storage unit 230, and a third control unit 330. The third storage unit 230 includes an image recognition library storage unit 231 and a master information storage unit 232. The third control unit 330 includes an image recognition processing unit 331. The image recognition processing unit 331 includes a feature extraction unit 332 and a master information selection unit 333.

[0041] The third communication unit 130 communicates with other devices. In particular, the third communication unit 130 receives an edited target image from the Web server device 22. The Web server device 22 also transmits to the Web server device 22 master information selected according to the edited target image. Moreover, the third communication unit 130 receives master information for each pattern of the metal nameplate from the database server device 25.

[0042] The third storage unit 230 stores various data. The function of the third storage unit 230 is executed using a storage device included in the image recognition server device 23. The image identification library storage unit 231 stores an image identification library. The image identification library is a program component for identifying the pattern of the target nameplate shown in the processed target image. The master information storage unit 232 stores master information for each pattern of the metal nameplate.

[0043] The third control unit 330 executes various processes by controlling each unit of the image identification server device 23. The function of the third control unit 330 is executed by a CPU included in the image identification server device 23 reading out a program from the third storage unit 230 and executing it. The image identification processing unit 331 selects master information corresponding to the processed target image from the master information stored in the master information storage unit 232 for each pattern of the metal nameplate.

[0044] The feature amount extraction unit 332 extracts feature amounts of the processed target image. The feature amount extraction unit 332 executes the image recognition library stored in the image recognition library storage unit 231 to extract feature amounts of the processed target image. The master information selection unit 333 selects master information corresponding to the pattern of the target image from the master information for each pattern of the metal nameplate stored in the master information storage unit 232 based on the feature amount extracted by the feature amount extraction unit 332 .

[0045] Specifically, for each piece of master information, the master information selection unit 333 calculates a similarity between the feature included in the master information and the feature calculated by the feature extraction unit 332. For the calculation of the similarity, a known similarity such as the L1 norm, the L2 norm, or the L∞ norm can be used. The master information selection unit 333 selects the master information having the highest similarity in feature amount as the master information according to the pattern of the target image. The image recognition server device 23, which includes the feature extraction unit 332 and the master information selection unit 333, is an example of an image recognition device.

[0046] The printed character reading server device 24 reads the printed characters from the processed target image. FIG. 6 is a schematic block diagram showing an example of the functional configuration of the printed character reading server device 24. As shown in FIG. 6, printed character reading server device 24 includes a fourth communication unit 140, a fourth storage unit 240, and a fourth control unit 340. Fourth storage unit 240 includes a character reading library storage unit 241 and a font storage unit 242. Fourth control unit 340 includes a printed character reading processing unit 341.

[0047] The fourth communication unit 140 communicates with other devices. In particular, the fourth communication unit 140 receives the processed target image from the Web server device 22. In addition, the fourth communication unit 140 transmits the printed characters read from the processed target image to the Web server device 22. The fourth storage unit 240 stores various data. The function of the fourth storage unit 240 is executed using a storage device included in the printed character reading server device 24.

[0048] The character reading library storage unit 241 stores a character reading library. The character reading library is, for example, a combination of an OCR engine and information used to operate the OCR engine, such as a dictionary. The font storage unit 242 stores fonts used for printing characters on the target nameplate. It is expected that the OCR engine can read characters with high accuracy by using the fonts stored in the font storage unit 242. The font storage unit 242 may be configured as a part of the character reading library storage unit 241.

[0049] Fourth control unit 340 executes various processes by controlling each unit of printed character reading server device 24. The functions of fourth control unit 340 are executed by a CPU included in printed character reading server device 24 reading a program from fourth storage unit 240 and executing the program. Printed character reading processor 341 reads printed characters from the processed target image. By executing the character reading library stored in character reading library storage unit 241, OCR is applied to the processed target image to read the printed characters. The arrangement of the printed characters in the processed target image is indicated by the master information. Therefore, printed character reading processor 341 identifies the position of the printed characters in the processed target image (the position of the area where the printed characters are written) based on the master information, and performs OCR on the identified position. When printed characters are written in multiple positions on the processed target image, printed character reading processing unit 341 reads the printed characters from each of the multiple positions where the printed characters are written.

[0050] The combination of the embossed character reading processor 312 and the printed character reading processor 341 corresponds to an example of a character reading unit. The embossed character reading processor 312 reads embossed characters by pattern matching, while the printed character reading processor 341 reads printed characters by using OCR, thus performing different character recognition processes for embossed characters and printed characters to read characters written on the target nameplate. The combination of the embossed character reading processor 312 and the printed character reading processor 341 reads both embossed characters and printed characters from one target image. Moreover, the combination of the embossed character reading server device 21 having the embossed character reading processing unit 312 and the printed character reading server device 24 having the printed character reading processing unit 341 corresponds to an example of a character reading device.

[0051] The database server device 25 stores the master information. In addition, the database server device 25 transmits the master information to the image recognition server device 23. FIG. 7 is a schematic block diagram showing an example of the functional configuration of the database server device 25. As shown in FIG. In the configuration shown in FIG. 7, the database server device 25 includes a fifth communication unit 150, a fifth storage unit 250, and a fifth control unit 350. The fifth storage unit 250 includes a master information storage unit 251 . The fifth control unit 350 includes a master information generating unit 351. The master information generating unit 351 includes a feature extracting unit 352.

[0052] The fifth communication unit 150 communicates with other devices. In particular, the fifth communication unit 150 acquires the master image, the layout information of the embossed characters and the printed characters in the master image, and the processed master image from the management terminal device 31 via the Web server device 22. Furthermore, the fifth communication unit 150 transmits the master information to the image recognition server device 23 .

[0053] The fifth storage unit 250 stores various data. The function of the fifth storage unit 250 is executed using a storage device included in the database server device 25. The master information storage unit 251 stores master information for each pattern of the metal nameplate. The fifth control unit 350 executes various processes by controlling each unit of the database server device 25. The function of the fifth control unit 350 is executed by a CPU included in the database server device 25 reading a program from the fifth storage unit 250 and executing the program.

[0054] The master information generating unit 351 generates master information for each pattern of the metal nameplate. For example, the master information generating unit 351 acquires the master image, the arrangement information of the embossed characters and the printed characters in the master image (the arrangement information of the embossed characters and the printed characters on the master nameplate), and the processed master image from the management terminal device 31 via the Web server device 22. Then, the master information generating unit 351 extracts the features of the processed master image in the feature extracting unit 352. The master information generating unit 351 generates master information by linking the master image, the feature, and the arrangement information of the embossed characters and the printed characters. Furthermore, the master information generating unit 351 modifies and deletes the master information stored in the feature extracting unit 352 in accordance with a user operation on the management terminal device 31 .

[0055] The feature extraction unit 352 extracts the feature of the processed master image. The feature extraction unit 352 extracts the feature of the processed master image by applying the same feature extraction process to the processed master image as the feature extraction process applied to the processed target image by the feature extraction unit 332 of the image recognition server device 23.

[0056] The master information storage unit 251 and the master information storage unit 232 of the image recognition server device 23 may store the same master information. For example, when the master information generation unit 351 updates (adds, changes, or deletes) the master information stored in the master information storage unit 251, difference information for updating the master information may be transmitted to the image recognition server device 23 via the fifth communication unit 150 by background processing.

[0057] The management terminal 31 is a terminal for the manager of the character reading system 1 . FIG. 8 is a schematic block diagram showing an example of the functional configuration of the management terminal device 31. As shown in FIG. In the configuration shown in FIG. 8, the management terminal device 31 includes a sixth communication unit 160, a sixth storage unit 260, a sixth control unit 360, a display unit 460, and an operation input unit 560. The sixth storage unit 260 includes an image processing library storage unit 261 . The sixth control unit 360 includes a master image acquisition unit 361 and an image processing unit 362 .

[0058] The sixth communication unit 160 communicates with other devices. In particular, the sixth communication unit 160 transmits to the Web server device 22 the master image, the layout information of the embossed characters and the printed characters in the master image, and the processed master image. The sixth storage unit 260 stores various data. The function of the sixth storage unit 260 is executed using a storage device included in the management terminal device 31. The image processing library storage unit 261 stores the same image processing library as the image processing library stored in the image processing library storage unit 211 .

[0059] The sixth control unit 360 executes various processes by controlling each unit of the management terminal device 31. The functions of the sixth control unit 360 are executed by the CPU included in the management terminal device 31 reading out a program from the sixth storage unit 260 and executing it. The master image acquisition unit 361 acquires a captured image of a metal nameplate. Specifically, the master image acquisition unit 361 acquires a captured image of a metal nameplate of a pattern for which the master information storage unit 251 of the database server device 25 does not store master information, such as when a metal nameplate of a new pattern is generated.

[0060] The method by which the master image acquisition unit 361 acquires the captured image of the metal nameplate is not limited to a specific method. For example, the captured image of the metal nameplate captured by the imaging device 11 may be stored in the Web server device 22 or the database server device 25. Then, a user such as an administrator of the character reading system 1 may select a captured image of a pattern whose master information has not yet been registered in the master information storage unit 251 of the database server device 25 from the captured images of the metal nameplate stored in the Web server device 22 or the database server device 25. In this case, the master image acquisition unit 361 acquires the selected captured image. Alternatively, a user such as an administrator of the character reading system 1 may obtain an image of the metal nameplate using, for example, a camera and have the management terminal device 31 read the image. In this case, the master image acquisition unit 361 acquires the image that the user has caused the management terminal device 31 to read.

[0061] The image processing unit 362 performs image processing on the captured image of the metal nameplate. The image processing unit 362 performs the same image processing on the captured image of the metal nameplate as the image processing unit 311 of the stamped character reading server device 21 performs on the target image, such as performing binarization processing and edge extraction on the captured image of the metal nameplate. The image processing unit 362 performs image processing on the captured image of the metal nameplate by executing the image processing library stored in the image processing library storage unit 261.

[0062] The display unit 460 includes a display screen such as a liquid crystal panel or an LED (Light Emitting Diode), and displays various images. In particular, the display unit 460 displays a master image. The operation input unit 560 includes input devices such as a keyboard and a mouse, and receives user operations. In particular, when the display unit 460 displays a master image, the operation input unit 560 receives user operations indicating the arrangement of the embossed characters and the printed characters in the master image.

[0063] Next, the operation of the character reading system 1 will be described with reference to FIGS. FIG. 9 is a diagram showing an example of a processing procedure in which the character reading system 1 generates and registers master information. 9, the master image acquisition unit 361 of the management terminal device 31 acquires a master image (sequence S111). The display unit 460 displays the master image acquired by the master image acquisition unit 361 (sequence S112). Then, the operation input unit 560 accepts a user operation indicating the arrangement of the embossed characters and the printed characters in the master image displayed by the display unit 460 (sequence S113).

[0064] Furthermore, the image processing unit 362 performs image processing on the master image acquired by the master image acquiring unit 361 to acquire a processed master image (sequence S114). The sixth communication unit 160, under the control of the sixth control unit 360, transmits the master image, the placement information of the embossed characters and the printed characters in the master image, and the processed master image to the Web server device 22, which then transfers them to the database server device 25 (sequence S115).

[0065] In the database server device 25, the feature extraction unit 352 extracts the feature of the processed master image from the management terminal device 31 (sequence S121). The master information generation unit 351 generates master information linking the master image, the arrangement information of the embossed characters and the printed characters in the master image, and the feature of the processed master image (sequence S122).

[0066] The master information storage unit 251 stores the master information under the control of the fifth control unit 350 (sequence S123). Furthermore, the fifth communication unit 150 transmits the master information to the image recognition server device 23 under the control of the fifth control unit 350 (sequence S124). In image recognition server device 23, master information storage unit 251 stores the master information from database server device 25 under the control of third control unit 330 (sequence S131).

[0067] FIG. 10 is a diagram showing an example of a processing procedure in which the character reading system 1 selects master information corresponding to a target image. In the process of FIG. 10, the imaging device 11 captures an image of the target nameplate at the site (sequence S211). The imaging device 11 outputs the obtained target image to the mobile terminal device 12, and the mobile terminal device 12 transfers it to the embossed character reading server device 21 (sequence S212).

[0068] In the embossed character reading server device 21, the image processing unit 311 performs image processing on the target image from the imaging device 11 (sequence S221). First communication unit 110 transmits the obtained processed target image to Web server device 22 under the control of first control unit 310, and Web server device 22 transfers it to image recognition server device 23 (sequence S222).

[0069] In the image recognition server device 23, the feature extraction unit 332 extracts the feature of the processed target image from the embossed character reading server device 21 (sequence S231). Based on the obtained feature, the master information selection unit 333 selects master information corresponding to the processed target image from the master information stored in the master information storage unit 232 for each pattern of the metal nameplate (sequence S232). The third communication section 130, under the control of the third control section 330, transmits the selected master information to the Web server device 22 (sequence S233).

[0070] FIG. 11 is a diagram showing an example of a processing procedure in which the character reading system 1 reads printed characters and embossed characters from a processed target image using master information. In the example of FIG. 11, the second communication unit 120 of the Web server device 22 transmits the processed target image and the master information to the printed character reading server device 24 under the control of the second control unit 320 (sequence S311).

[0071] In the printed character reading server device 24, the printed character reading processor 341 reads the printed characters from the processed target image based on the layout of the printed characters indicated by the master information (sequence S321). The fourth communication unit 140 transmits the result of reading the printed characters from the processed target image to the Web server device 22 under the control of the fourth control unit 340 (sequence S322).

[0072] In the Web server device 22, the reference processing unit 321 cuts out images of the positions of the embossed characters and the printed characters indicated by the master information from the target image (sequence S331). The second communication unit 120 transmits the master information, text data of the read printed characters (the result of reading the printed characters by the printed character reading processing unit 341), and the cut-out image from the target image to the embossed character reading server device 21 under the control of the second control unit 320 (sequence S332).

[0073] In the embossed character reading server device 21, the embossed character reading processor 312 reads the embossed characters from the processed target image based on the embossed character arrangement information indicated by the master information (sequence S341). The first communication unit 110 transmits text data of the read printed characters (printed character reading result), text data of the read embossed characters (embossed character reading result), and an image cut out from the target image to the mobile terminal device 12 under the control of the first control unit 310 (sequence S342).

[0074] The portable terminal device 12 displays the results of reading the printed characters and the results of reading the embossed characters alongside the corresponding cut-out images (sequence S351). An operator checking the results of reading can compare the results of reading with the cut-out images to determine whether the results of reading are correct or not. The mobile terminal device 12 accepts a correction operation for the displayed read result (printed characters and embossed characters), and when a correction operation is performed, corrects the read result in accordance with the operation (sequence S352). The obtained stamped and printed characters can be used, for example, by a worker when writing a work report as information about the transformer being worked on.

[0075] As described above, the first communication unit 110 of the embossed character reading server device 21 acquires the target image. As described above, the target image is an image of an outdoor metal nameplate installed on a transformer, and is an image of the metal nameplate on which embossed characters and printed characters are written. The image processing unit 311 performs image processing on the target image. The feature amount extraction unit 332 of the image recognition server device 23 extracts feature amounts from the target image to which the image processing has been applied. The master information selection unit 333 selects master information corresponding to the pattern of the target image from among the master information provided for each pattern of the metal nameplate, based on the feature extracted by the feature extraction unit 332. The master information is information in which the feature extracted from the processed master image in which the same image processing as that for the target nameplate is applied to the master image is linked to the arrangement information of the stamped characters and the printed characters. The master image is an image of the metal nameplate selected for each pattern of the metal nameplate. The combination of the embossed character reading processing unit 312 and the printed character reading processing unit 341 performs different character recognition processes for embossed characters and printed characters on the processed target image based on the character arrangement information indicated by the selected master information, thereby reading the characters written on the target nameplate. The character reading system 1 can read both embossed and printed characters.

[0076] Furthermore, the combination of the embossed character reading processor 312 and the printed character reading processor 341 reads both the embossed characters and the printed characters from one target image. According to the character reading system 1, the photographer only needs to take an image of one metal nameplate once, which reduces the burden on the photographer.

[0077] Moreover, the image processing unit 311 performs binarization processing and edge extraction. According to the character reading system 1, for example, the contours of an out-of-focus image can be made clearer, and in this respect, characters can be read with high accuracy.

[0078] Next, the configuration of an embodiment of the present invention will be described with reference to FIGS. Fig. 12 is a diagram showing an example of the configuration of a character reading system according to an embodiment. A character reading system 600 shown in Fig. 12 includes a target image acquisition unit 601, an image processing unit 602, a feature extraction unit 603, a master information selection unit 604, and a character reading unit 605.

[0079] In this configuration, the target image acquisition unit 601 acquires a target image, which is an image of an outdoor metal nameplate installed on a transformer, and is an image of a metal nameplate on which stamped characters and printed characters are written. The image processing unit 602 performs image processing on the target image. The feature extraction unit 603 extracts features from the target image to which the image processing has been applied. The master information selection unit 604 selects master information corresponding to the pattern of the target image based on the features, from among master information provided for each pattern of the metal nameplate, in which the features extracted from the master image, which is an image of the metal nameplate to which the image processing has been applied, and the arrangement information of the stamped characters and the printed characters are linked. The character reading unit 605 performs character recognition processing that differs between the stamped characters and the printed characters on the target image to which the image processing has been applied, based on the character arrangement information indicated by the selected master information, to read the characters written on the target nameplate. The character reading system 600 can read both embossed and printed characters.

[0080] Fig. 13 is a diagram showing another example of the configuration of a character reading system according to an embodiment. A character reading system 610 shown in Fig. 13 includes an imaging device 611, an image processing device 612, an image recognition device 613, and a character reading device 616. The image recognition device 613 includes a feature extraction unit 614 and a master information selection unit 615.

[0081] In this configuration, the imaging device 611 captures a target image, which is an image of an outdoor metal nameplate installed on a transformer, and which is an image of a metal nameplate with stamped characters and printed characters. The image processing device 612 performs image processing on the target image. The feature extraction unit 614 extracts features from the target image to which the image processing has been applied. The master information selection unit 615 selects master information corresponding to the pattern of the target image based on the features, from among master information provided for each pattern of the metal nameplate, which is a metal nameplate image to which image processing has been applied and master information in which the features extracted from the master image and the arrangement information of the stamped characters and printed characters are linked. The character reading system 610 can read both embossed and printed characters.

[0082] FIG. 14 is a flowchart showing an example of a processing procedure in the character reading method according to the embodiment. The process shown in FIG. 14 includes a target image acquisition step (step S11), an image processing step (step S12), a feature amount extraction step (step S13), a master information selection step (step S14), and a character reading step (step S15).

[0083] In the target image acquisition step (step S11), a target image is acquired, which is an image of a metal nameplate installed outdoors on a transformer, and which is an image of a metal nameplate with stamped characters and printed characters. In the image processing step (step S12), image processing is performed on the target image. In the feature extraction step (step S13), feature values ​​are extracted from the target image to which image processing has been applied. In the master information selection step (step S14), master information corresponding to the pattern of the target image is selected based on feature values ​​from among master information provided for each pattern of the metal nameplate, which is an image of the metal nameplate to which image processing has been applied, and master information in which feature values ​​extracted from the master image, which is an image of the metal nameplate to which image processing has been applied, and arrangement information of the stamped characters and the printed characters are linked. In the character reading step (step S15), character recognition processing that differs between the stamped characters and the printed characters is performed on the target image to which image processing has been applied, based on the character arrangement information indicated by the selected master information, to read the characters written on the target nameplate. According to the character reading method shown in FIG. 14, both engraved characters and printed characters can be read.

[0084] FIG. 15 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. In the configuration shown in FIG. 15, a computer 700 includes a CPU (Central Processing Unit) 710, a main memory device 720, an auxiliary memory device 730, and an interface 740. Any one or more of the embossed character reading server device 21, Web server device 22, image recognition server device 23, printed character reading server device 24, database server device 25, management terminal device 31, and character reading system 600 may be implemented in the computer 700. In this case, the operation of each of the above-mentioned processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730 and loads it in the main storage device 720, and executes the above-mentioned processing according to the program. The CPU 710 also secures storage areas in the main storage device 720 corresponding to each of the above-mentioned storage units according to the program.

[0085] When the embossed character reading server device 21 is implemented in the computer 700, the operations of the first control unit 310 and each of its units are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program. Furthermore, the CPU 710 allocates in the main memory 720 a storage area corresponding to the first storage unit 210 and each of its units in accordance with the program. The communication by the first communication unit 110 is executed by the interface 740 having a communication function and performing communication under the control of the CPU 510.

[0086] When the Web server device 22 is implemented in the computer 700, the operations of the second control unit 320 and each of its units are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program. Furthermore, the CPU 710 allocates a memory area corresponding to the second memory unit 220 in the main memory device 720 in accordance with the program. The communication by the second communication unit 120 is executed by the interface 740 having a communication function and performing communication under the control of the CPU 710.

[0087] When the image identification server device 23 is implemented in the computer 700, the operations of the third control unit 330 and each of its units are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program. Furthermore, the CPU 710 allocates in the main memory 720 a storage area corresponding to the third storage unit 230 and each of its units in accordance with the program. The communication by the third communication unit 130 is executed by the interface 740 having a communication function and performing communication under the control of the CPU 710.

[0088] When printed character reading server device 24 is implemented in computer 700, the operations of fourth control unit 340 and each of its units are stored in the form of a program in auxiliary storage device 730. CPU 710 reads the program from auxiliary storage device 730, loads it into main storage device 720, and executes the above-mentioned processing in accordance with the program. Furthermore, the CPU 710 allocates in the main memory 720 a storage area corresponding to the fourth storage unit 240 and each of its units in accordance with the program. The communication by the fourth communication unit 140 is executed by the interface 740 having a communication function and performing communication under the control of the CPU 710.

[0089] When the database server device 25 is implemented in the computer 700, the operations of the fifth control unit 350 and each of its units are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing in accordance with the program. Furthermore, the CPU 710 allocates in the main memory 720 a fifth memory unit 250 and memory areas corresponding to each unit thereof in accordance with the program. The communication by the fifth communication unit 150 is executed by the interface 740 having a communication function and performing communication under the control of the CPU 710.

[0090] When the management terminal device 31 is implemented in the computer 700, the operations of the sixth control unit 360 and each of its units are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program. Furthermore, the CPU 710 allocates in the main memory 720 a sixth memory unit 260 and memory areas corresponding to each unit thereof in accordance with the program. Communication by the sixth communication unit 160 is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Display by the display unit 460 is performed by the interface 740 having a display screen and performing display under the control of the CPU 710. Reception of a user operation by the operation input unit 560 is performed by the interface 740 having an input device and receiving the user operation.

[0091] When the character reading system 600 is implemented in a computer 700, the operations of the target image acquisition unit 601, the image processing unit 602, the feature extraction unit 603, the master information selection unit 604, and the character reading unit 605 are stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing according to the program.

[0092] Note that programs for implementing all or part of the functions of embossed character reading server device 21, Web server device 22, image recognition server device 23, printed character reading server device 24, database server device 25, management terminal device 31, and character reading system 600 may be recorded on a computer-readable recording medium, and the programs recorded on the recording medium may be read into a computer system and executed to process each part. The term "computer system" here includes the OS (operating system) and hardware such as peripheral devices. "Computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into computer systems. The above program may be for realizing part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system.

[0093] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0094] 1, 600, 610 Character Reading System 11, 611 Imaging device 12 Mobile terminal device 21. Stamp character reading server device 110 First Communications Department 210 First memory section 211 Image processing library storage unit 310 First Control Section 311, 362, 602 Image processing unit 312 Stamp character reading processing unit 22 Web server equipment 120 Second Communications Department 220 Second memory section 320 Second Control Section 321 Enquiry Processing Unit 322 Master Management Department 23 Image Identification Server Device 130 Third Communications Department 230 Third Memory 231 Image Identification Library Storage Unit 232 Master information storage unit 330 Third Control Section 331 Image Recognition Processing Unit 332, 352, 603, 614 Feature Extraction Unit 333, 604, 615 Master information selection section 24 Printed character reading server device 140 Fourth Communication Department 240 Fourth Memory 241 Character reading library memory section 242 Font storage section 340 Fourth Control Section 341 Printed character reading processing unit 25 Database server device 150 Fifth Communication Department 250 Fifth Memory 251 Master Information Storage Unit 350 Fifth Control Section 351 Master Information Generation Unit 31 Management terminal 160 Sixth Communication Department 260 Sixth Memory 261 Image Processing Library Storage Unit 360 6th Control Section 361 Master Image Acquisition Unit 460 Display section 560 Operation input section 601 Target image acquisition unit 605 Character Reading Unit 612 Image Processing Device 613 Image Recognition Device 616 Character reader

Claims

1. a target image acquisition unit that acquires a target image, which is an image of a metal nameplate installed outdoors on a transformer, the metal nameplate having stamped characters and printed characters; an image processing unit that performs image processing on the target image; a feature extraction unit that extracts features from the target image to which the image processing has been applied; a master information selection unit that selects master information corresponding to the pattern of the target image based on the feature amount from among master information provided for each pattern of the metal nameplate, the master information being associated with feature amounts extracted from an image obtained by applying the image processing to a master image that is an image of the metal nameplate and arrangement information of the embossed characters and the printed characters; a character reading unit that performs character recognition processing that differs between embossed characters and printed characters on the target image to which the image processing has been applied based on character arrangement information indicated by the selected master information, and reads characters indicated in the target image; A character reading system comprising:

2. the character reading unit reads both the embossed characters and the printed characters from one target image; 2. The character reading system according to claim 1.

3. The image processing unit performs binarization processing and edge extraction.

3. A character reading system according to claim 1 or 2.

4. The system includes an imaging device, an image processing device, an image recognition device, and a character reading device, The imaging device captures a target image, which is an image of an outdoor metal nameplate installed on a transformer, the target image being an image of a metal nameplate on which stamped characters and printed characters are written; The image processing device performs image processing on the target image, The image identification device includes: a feature extraction unit that extracts features from the target image to which the image processing has been applied; a master information selection unit that selects master information corresponding to the pattern of the target image based on the feature amount from among master information provided for each pattern of the metal nameplate, the master information being associated with feature amounts extracted from an image obtained by applying the image processing to a master image that is an image of the metal nameplate and arrangement information of the embossed characters and the printed characters; Equipped with the character reading device performs character recognition processing, which differs between embossed characters and printed characters, on the target image to which the image processing has been applied, based on character arrangement information indicated by the selected master information, thereby reading the characters indicated in the target image. Character reading system.

5. acquiring a target image, which is an image of an outdoor metal nameplate installed on a transformer, the target image being an image of the metal nameplate having stamped characters and printed characters; performing image processing on the target image; extracting features from the target image to which the image processing has been applied; A step of selecting master information corresponding to the pattern of the target image based on the feature amount from among master information provided for each pattern of the metal nameplate, the master information being linked to feature amounts extracted from an image obtained by applying the image processing to a master image that is an image of the metal nameplate and arrangement information of the embossed characters and the printed characters; a step of performing character recognition processing, which differs between embossed characters and printed characters, on the target image to which the image processing has been applied based on character arrangement information indicated by the selected master information, thereby reading characters indicated in the target image; A character reading method comprising:

6. On the computer, acquiring a target image, which is an image of an outdoor metal nameplate installed on a transformer, the target image being an image of the metal nameplate having stamped characters and printed characters; performing image processing on the target image; extracting features from the target image to which the image processing has been applied; A step of selecting master information corresponding to the pattern of the target image based on the feature amount from among master information provided for each pattern of the metal nameplate, the master information being linked to feature amounts extracted from an image obtained by applying the image processing to a master image that is an image of the metal nameplate and arrangement information of the embossed characters and the printed characters; a step of performing character recognition processing, which differs between embossed characters and printed characters, on the target image to which the image processing has been applied based on character arrangement information indicated by the selected master information, thereby reading characters indicated in the target image; A program for executing the above.

Images