Character recognition device, character recognition method and character recognition program

Abstract

Provided are a character recognition device, a character recognition method, and a character recognition program for shortening the reading time when reading a character string through character recognition processing. The device comprises: an extraction unit for extracting a specific character string from a captured image; a determination unit for determining a processing area of ​​the captured image according to the position of the extracted specific character string; and a recognition unit for performing character recognition on the determined processing area in the captured image.

Description

Technical Field

[0001] The present disclosure relates to a character recognition device, a character recognition method and a character recognition program. Background Art

[0002] There is known an optical character recognition technology that uses a portable terminal or the like to photograph device nameplates installed on various devices on site and reads character strings through character recognition processing. According to this technology, for example, device data can be read from device nameplates attached to various devices.

[0003] [Prior art literature]

[0004] [Patent Literature]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2020-119195

[0006] [Patent Document 2] Japanese Patent Application Publication No. 2013-206131 Summary of the invention

[0007] [Problems to be solved by the invention]

[0008] On the other hand, the nameplate of the device contains many types of characters such as letters, numbers, symbols, and Chinese characters, and it takes a certain amount of time to read the character string through character recognition processing.

[0009] The present disclosure provides a character recognition device, a character recognition method, and a character recognition program for shortening a reading time when a character string is read through a character recognition process.

[0010]

Methods to solve the problem

[0011] A first aspect of the present disclosure provides a character recognition device comprising:

[0012] An extraction unit extracts a specific character string from the captured image;

[0013] a determination unit that determines a processing area of ​​the captured image based on a position of the extracted specific character string; and

[0014] The recognition unit performs character recognition on a processing area determined in the captured image.

[0015] According to the first aspect of the present disclosure, it is possible to provide a character recognition device that shortens the reading time when a character string is read by a character recognition process.

[0016] In addition, a second aspect of the present disclosure is a character recognition device according to the first aspect,

[0017] The character types of a first library used by the extraction unit in character recognition when extracting the specific character string are fewer than the character types of a second library used by the recognition unit in character recognition.

[0018] In addition, a third aspect of the present disclosure is the character recognition device described in the second aspect,

[0019] A first storage unit is provided for storing determination information defining the composition of characters included in the specific character string.

[0020] The extraction unit performs character recognition on the captured image using the first library, and determines a character string including characters specified by the determination information from among the recognized character recognition character strings, thereby extracting the specific character string.

[0021] In addition, a fourth aspect of the present disclosure is the character recognition device described in the third aspect,

[0022] The determination information specifies any one of the number of characters, the type of characters, and the order of characters as the composition of the characters.

[0023] In addition, a fifth aspect of the present disclosure is the character recognition device according to the fourth aspect,

[0024] The determination unit determines, as the processing area, an area that includes a bounding box used by the extraction unit when extracting the specific character string and has a height that is a predetermined multiple of the height of the bounding box and has a width from the position of the specific character string to the end position of the captured image.

[0025] In addition, a sixth aspect of the present disclosure is the character recognition device according to the first aspect,

[0026] A second storage unit for storing a list of identification information is provided.

[0027] The recognition unit determines whether the recognized character matches any identification information included in the identification information list.

[0028] In addition, a seventh aspect of the present disclosure is a character recognition device according to the sixth aspect,

[0029] An output unit is provided for outputting the extracted specific character string as data corresponding to the matching identification information when the recognition unit determines that the specific character string matches any identification information included in the identification information list.

[0030] In addition, an eighth aspect of the present disclosure is the character recognition device according to the seventh aspect,

[0031] When the plurality of processing areas partially overlap each other and the same identification information is included in the plurality of processing areas, the output unit outputs a specific character string closer to the position of the same identification information as data corresponding to the same identification information.

[0032] In addition, a ninth aspect of the present disclosure is a character recognition device according to the sixth aspect,

[0033] The identification information included in the identification information list is a label of the device, and includes at least one of a model name, a manufacturing number, a manufacturing year and month, a refrigerant type, a refrigerant amount, and a compressor capacity.

[0034] In addition, a tenth aspect of the present disclosure is the character recognition device according to the first aspect,

[0035] The captured image is an image obtained by capturing a nameplate attached to the equipment.

[0036] In addition, an eleventh aspect of the present disclosure is a character recognition method having:

[0037] Extraction process, extracting specific character strings from captured images;

[0038] a determining step of determining a processing area of ​​the captured image based on a position of the extracted specific character string; and

[0039] The recognition step performs character recognition on a processing area determined in the captured image.

[0040] According to the eleventh aspect of the present disclosure, it is possible to provide a character recognition method that shortens the reading time when a character string is read by a character recognition process.

[0041] In addition, a twelfth aspect of the present disclosure is a character recognition program for causing a computer to execute:

[0042] Extraction process, extracting specific character strings from captured images;

[0043] a determining step of determining a processing area of ​​the captured image based on a position of the extracted specific character string; and

[0044] The recognition step performs character recognition on a processing area determined in the captured image.

[0045] According to the twelfth aspect of the present disclosure, it is possible to provide a character recognition program that shortens the reading time when a character string is read by a character recognition process. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a diagram showing an example of use of a character recognition device.

[0047] Figure 2 This is a diagram showing an example of the hardware configuration of a character recognition device.

[0048] Figure 3 This is a diagram showing an example of the functional structure of a character recognition device.

[0049] Figure 4 It is a diagram showing the details of the functional structure of the extraction unit.

[0050] Figure 5 It is a diagram showing a specific example of the processing of the extraction unit.

[0051] Figure 6 This is a diagram showing the details of the functional structure of the determination unit.

[0052] Figure 7 This is a diagram showing a specific example of the processing of the determination unit.

[0053] Figure 8 It is a diagram showing the details of the functional structure of the recognition unit.

[0054] Fig. 9 It is a diagram showing a specific example of the processing of the recognition unit.

[0055] Fig.10 This is a first flowchart showing the flow of character recognition processing.

[0056] Fig.11 This is a second flow chart showing the flow of character recognition processing. DETAILED DESCRIPTION

[0057] Hereinafter, each embodiment will be described with reference to the drawings. In addition, in this specification and the drawings, the same reference numerals are given to components having substantially the same functional configuration, and repeated descriptions are omitted.

[0058] [First embodiment]

[0059] <Use Example of Character Recognition Device>

[0060] First, a usage example of the character recognition device according to the first embodiment will be described. Figure 1 2 is a diagram showing an example of using a character recognition device. Figure 1 As shown on the left side of FIG. 1 , the character recognition device 120 is used, for example, in a scenario where device data is read from a device nameplate 111 attached to a side surface of a field device 110 .

[0061] According to the character recognition device 120, for example, when the maintenance worker 130 conducts a field survey and registers the survey results in a server device (not shown), the maintenance worker 130 can save the time of manually inputting device data for specifying the field device 110 to be surveyed. In addition, the maintenance worker 130 can be prevented from inputting device data by mistake.

[0062] In addition, in this embodiment, the field device 110 is an outdoor unit of an air conditioner and the character recognition device 120 is a portable terminal. However, the field device 110 is not limited to the outdoor unit of the air conditioner, and may be a unit other than the outdoor unit of the air conditioner, or a field device other than the air conditioner. In addition, the character recognition device 120 is not limited to a portable terminal, and may be a terminal other than a portable terminal as long as it has an optical character recognition function.

[0063] Figure 1 The right side of shows the situation that the maintenance worker 130 photographed the equipment nameplate 111 and the photographed image 121 is displayed on the display screen of the character recognition device 120. Figure 1 As shown on the right side of , the captured image 121 includes an image area 122 corresponding to the device nameplate 111 .

[0064] When the field device 110 is an outdoor unit of an air conditioner, as shown in the image area 122, the following information is provided on the device nameplate 111, for example:

[0065] ·Indicates that it is an outdoor unit.

[0066] ·Indicates that it is an outdoor unit that performs both cooling and heating functions.

[0067] ·Records about the model (label, model code data),

[0068] · Record of manufacturing number (label, manufacturing number data),

[0069] Records on product quality (labels, quality data),

[0070] ·Records of power supply specifications (labels, phase data, voltage data, frequency data),

[0071] ·Records on the output of the compressor motor (label, power consumption data),

[0072] · Recording of management numbers (labels, management code data),

[0073] ·Records on the amount and type of refrigerant (label, refrigerant quantity data, refrigerant type code),

[0074] ·Record of design pressure instructions (label, pressure data),

[0075] ·JIS mark record,

[0076] ・Records on JIS (registered certification body code data, JIS code data).

[0077] Thus, the description on the device nameplate 111 includes many types of characters such as letters, numbers, symbols, and Chinese characters (the language of the country where the field device 110 is installed). Figure 1 Although not shown in the image area 122, the device nameplate 111 may include, in addition to the above-mentioned records, records regarding the year and month of manufacture (label, data regarding the year and month of manufacture), for example.

[0078] <Hardware Structure of Character Recognition Device>

[0079] Next, the hardware configuration of the character recognition device 120 will be described. Figure 2 FIG. 1 is a diagram showing an example of the hardware structure of a character recognition device. Figure 2 As shown, the character recognition device 120 includes a processor 201, a memory 202, an auxiliary storage device 203, a user interface device 204, a camera device 205, and a communication device 206. In addition, the hardware components of the character recognition device 120 are connected to each other via a bus 207.

[0080] The processor 201 includes various computing devices such as a CPU (Central Processing Unit), etc. The processor 201 reads various programs (for example, a character recognition program described later) to the memory 202 and executes them.

[0081] The memory 202 includes a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 201 and the memory 202 form a so-called computer, and the processor 201 executes various programs read out on the memory 202, so that the computer realizes various functions.

[0082] The auxiliary storage device 203 stores various programs and various data used when the various programs are executed by the processor 201 (for example, a first library, device data rules, a second library, a label list, etc. described later).

[0083] The user interface device 204 has, for example, an operation function for the maintenance worker 130 to perform various operations and a display function for displaying the processing results of various processes executed by the character recognition device 120 .

[0084] The imaging device 205 captures the device nameplate 111 and generates a captured image 121. The communication device 206 is a communication device for communicating with an external device (for example, a server device not shown) via a network.

[0085] In addition, various programs installed in the auxiliary storage device 203 are installed by being downloaded from a network via the communication device 206, for example.

[0086] <Functional Structure of Character Recognition Device>

[0087] Next, the functional structure of the character recognition device 120 will be described. As described above, a character recognition program is installed in the character recognition device 120, and by executing the program, the character recognition device 120 functions as follows:

[0088] Extraction unit 310,

[0089] ·Determination unit 320,

[0090] Identification unit 330.

[0091] The extraction unit 310 acquires a captured image (eg, captured image 121) captured by the imaging device 205. The extraction unit 310 also performs character recognition on the acquired captured image using the first library stored in the first library storage unit 340, and recognizes a character string from the recognized characters.

[0092] Furthermore, the extraction unit 310 extracts a candidate of device data (specific character string) by determining a character string having a structure of characters specified by the device data rule (determination information) stored in the device data rule storage unit 350 (an example of the first storage unit) among the recognized character strings.

[0093] The extraction unit 310 also notifies the identification unit 330 of the extracted candidate device data (specific character string). The extraction unit 310 also notifies the determination unit 320 of the information (height, width, and position in the captured image) of the bounding box used when extracting the device data candidate (specific character string).

[0094] In addition, the first library stored in the first library storage unit 340 is composed of, for example, alphanumeric characters, and is a library having fewer character types than a general character recognition library. The general character recognition library referred to here refers to a library that includes, in addition to alphanumeric characters, characters and numbers representing the language of the country where the field device 110 is installed, characters used for device nameplates (for example, Greek numerals, Roman letters, etc.), various symbols, and the like.

[0095] In the present embodiment, the device data rule stored in the device data rule storage unit 350 is information that specifies the character structure of the device data of the outdoor unit of the air conditioner. In addition, the device data of the outdoor unit of the air conditioner includes, for example, the following:

[0096] Model code data,

[0097] Manufacturing number data,

[0098] Manufacturing year and month data,

[0099] Refrigerant type code,

[0100] Refrigerant quantity data,

[0101] Power consumption data of compressor motor.

[0102] That is, the extraction unit 310 uses the alphanumeric characters constituting the first library to perform character recognition on the character area within the image area 122 of the captured image 121. In addition, the extraction unit 310 recognizes the recognized alphanumeric character string and determines the character string that contains the characters specified by the device data rule among the recognized character strings. Thus, the extraction unit 310 extracts the candidate of the device data of the outdoor unit of the air conditioner and notifies the recognition unit 330.

[0103] The decision unit 320 obtains the captured image captured by the camera 205 and obtains the bounding box information (height, width, and position in the captured image) notified by the extraction unit 310. In addition, the decision unit 320 determines the processing area of ​​the captured image based on the obtained bounding box information and notifies the recognition unit 330.

[0104] The processing area determined by the determination unit 320 is a target area for character recognition by the recognition unit 330 , and is a target area for searching for a “label” (an example of identification information) corresponding to a candidate of device data.

[0105] The recognition unit 330 acquires the processing area determined by the determination unit 320 , performs character recognition on the acquired processing area using the second library stored in the second library storage unit 360 , and recognizes a character string from the recognized characters.

[0106] The second library stored in the second library storage unit 360 is the above-mentioned general library for character recognition.

[0107] In addition, the recognition unit 330 refers to the label list (an example of the identification information list) held in the label list holding unit 370 (an example of the second holding unit) to determine whether the character string recognized in the processing area matches any label included in the label list. When the recognition unit 330 determines that it matches any label included in the label list, it associates the candidate of the device data with the label and outputs the device data as the matching label.

[0108] In other words, the recognition section 330 specifies a “label” corresponding to a candidate for device data from among the character strings recognized in the processing area.

[0109] Thus, in the character recognition device 120, when reading the device data from the device nameplate 111, instead of performing character recognition on all of the many character types, the following method is adopted:

[0110] The processing area is narrowed down by performing character recognition using the first library with fewer character types and extracting device data candidates using device rule data that specifies the structure of characters. Then, character recognition is performed using the second library with more character types for the narrowed down processing area.

[0111] For the character string recognized by performing character recognition using the second library, it is determined whether it matches the label using the label list, and if it is determined to match, the candidate of the device data is output as the device data of the matching label.

[0112] Thus, according to the first embodiment, the reading time when the device data is read from the device nameplate can be shortened by the character recognition process. In addition, according to the first embodiment, the reading accuracy can be improved.

[0113] <Details of the functional configuration of the extraction unit>

[0114] Next, the functional configuration of the extraction unit 310 will be described in detail. Figure 4 is a diagram showing the details of the functional structure of the extraction unit. Figure 4 As shown, the extraction unit 310 includes a character string recognition unit 410 , a specific character string determination unit 420 , and a bounding box determination unit 430 .

[0115] The character string recognition unit 410 acquires the image captured by the camera 205 , performs character recognition using the first library stored in the first library storage unit 340 , and recognizes a character string from the recognized alphanumeric characters.

[0116] The specific character string determination unit 420 extracts a candidate of device data by determining whether the character string recognized by the character string recognition unit 410 is a character string composed of characters specified by the device data rule.

[0117] The device data rule 440 specifies the character structure (number of characters, character type, character order, etc.) of the device data of the outdoor unit of the air conditioner. According to the device data rule 440, the character structure of the "model code data" is "more than 6 characters and less than 11 characters, and more than 1 uppercase English and numeric characters and more than 2 numeric characters". Therefore, in the specific character string determination unit 420, the character string containing this character structure is determined as a candidate for the "model code data" and extracted.

[0118] In addition, according to the device data rule 440, the character composition of the "manufacturing number data" is defined as "(more than 6 characters and less than 12 characters, and the first character is an uppercase English number)" or "(more than 6 characters and less than 12 characters, and all are numbers)". Therefore, in the specific character string determination unit 420, the character string containing the character composition is determined as "manufacturing number data" and extracted.

[0119] In addition, according to the device data rule 440, the character structure of "manufacturing year and month data" is defined as "containing ' / ', and the characters before and after the ' / ' are more than 2 characters, and the characters before and after the ' / ' are within 2 characters, and the manufacturing month is an integer less than 12". Therefore, in the specific character string determination unit 420, the character string containing the character structure is determined as a candidate for "manufacturing year and month data" and extracted.

[0120] In addition, according to the device data rule 440, the character structure of the "refrigerant type code" is defined as "five characters or less, the first character is 'G', and the number of digits is two or more". Therefore, in the specific character string determination unit 420, the character string containing this character structure is determined as a candidate for the "refrigerant type code" and extracted.

[0121] In addition, according to the device data rule 440, the character structure of "refrigerant quantity data" is defined as "including any of 'kg', 'ke', and 'k' at the same height, including the type of refrigerant at the same height, and including '.' and a number". Therefore, in the specific character string determination unit 420, a character string including the character structure is determined as a candidate for "refrigerant quantity data" and extracted.

[0122] In addition, according to the machine data rule 440, the character structure of "power consumption data of the compressor motor" is defined as "including either 'kw' or 'km' at the same height, and including '.' and a number". Therefore, in the specific character string determination unit 420, the character string including the character structure is determined as a candidate for "power consumption data of the compressor motor" and extracted.

[0123] The specific character string determination unit 420 notifies the determined device data candidates to the recognition unit 330. Specifically, the specific character string determination unit 420 notifies the recognition unit 330 of the model code data candidates, the manufacturing number data candidates, the manufacturing year and month data candidates, the refrigerant type code candidates, the refrigerant amount data candidates, the compressor motor power consumption data candidates, and the like.

[0124] Furthermore, the specific character string determination unit 420 notifies the bounding box determination unit 430 of the bounding box used when extracting the determined device data candidates.

[0125] The bounding box determination unit 430 calculates the height, width, and position in the captured image based on the acquired bounding box, and notifies the determination unit 320 of the bounding box information (height, width, and position in the captured image).

[0126] <Specific Example of Processing by the Extraction Unit>

[0127] Next, a specific example of the processing of the extraction unit 310 will be described. Figure 5 It is a diagram showing a specific example of the processing of the extraction unit.

[0128] in, Figure 5 5 a denotes an image region 122 corresponding to the device nameplate 111 extracted and displayed from the captured image 121 acquired by the character string recognition unit 410 .

[0129] in addition, Figure 5 The case represented by 5b is a case where the character string recognition unit 410 performs character recognition using the first library and recognizes an alphanumeric character string (symbols 501 to 512).

[0130] in addition, Figure 5 5c shows a case where the specific character string determination unit 420 determines and extracts the character strings indicated by reference numerals 501, 502, 505 to 508, and 512 as character strings including the characters defined by the device data rule in the recognized character string.

[0131] Furthermore, the bounding box determination unit 430 acquires a bounding box used when extracting the character string determined by the specific character string determination unit 420 (character strings indicated by reference numerals 501 , 502 , 505 to 508 , and 512 ).

[0132] <Details of the functional structure of the determination unit>

[0133] Next, the functional configuration of the determination unit 320 will be described in detail. Figure 6 is a diagram showing the details of the functional structure of the decision unit. Figure 6 As shown, the determination unit 320 includes a height information calculation unit 610 , a width information calculation unit 620 , and a processing region determination unit 630 .

[0134] The height information calculation unit 610 obtains the bounding box information (height, width, and position in the captured image) notified by the bounding box determination unit 430, and multiplies the obtained bounding box height information by a predetermined multiple to calculate the height information of the processing area. In this way, the height information calculation unit 610 determines the position of the bottom side and the position of the top side of the processing area.

[0135] The width information calculation unit 620 obtains the bounding box information (height, width, and position in the captured image) notified by the bounding box determination unit 430, and expands the obtained bounding box to the end in the width direction to calculate the width information of the processing area. Thus, the width information calculation unit 620 determines the right position and the left position of the processing area.

[0136] In addition, the end in the width direction refers to the left end position or the right end position of the captured image 121, and whether the width information calculation unit 620 expands to the left end position or the right end position in the width direction depends on the writing direction of the device nameplate 111. For example, when the language of the device nameplate 111 that writes "label" is a language that writes from left to right, the width information calculation unit 620 expands the bounding box to the left end position of the captured image. On the other hand, when the language of the device nameplate 111 that writes "label" is a language that writes from right to left, the width information calculation unit 620 expands the bounding box to the right end position of the captured image.

[0137] The processing region determination unit 630 determines the processing region based on the height information calculated by the height information calculation unit 610 and the width information calculated by the width information calculation unit 620. The determined processing region is extracted from the captured image 121 and the extracted processing region is notified to the recognition unit 330.

[0138] <Specific example of processing by the determination unit>

[0139] Next, a specific example of the processing of the determination unit 320 will be described. Figure 7 This is a diagram showing a specific example of the processing of the determination unit.

[0140] in, Figure 7 7 a shows each bounding box (bounding boxes 701 to 707 ) in the image area 122 of the captured image 121 notified by the bounding box determination unit 430 .

[0141] Figure 7 7b in FIG. 7 shows a state where a processing area is determined for the bounding box 701 and the bounding box 707 among the bounding boxes 701 to 707 .

[0142] Here, the processing area 710 indicates a processing area determined by multiplying the bounding box 701 by a predetermined multiple in the height direction and expanding it in the width direction to the end position 700 of the captured image 121 .

[0143] Similarly, the processing area 720 indicates a processing area determined by increasing the size of the bounding box 707 by a predetermined multiple in the height direction and expanding it in the width direction to the end position 700 of the captured image 121 .

[0144] <Details of the functional structure of the identification unit>

[0145] Next, the functional configuration of the recognition unit 330 will be described in detail. Figure 8 FIG. 4 is a diagram showing the details of the functional structure of the recognition unit. Figure 8 As shown, the recognition unit 330 includes a processing area character recognition unit 810 and a determination unit 820 .

[0146] The processing area character recognition unit 810 obtains the processing area from the processing area determination unit 630, performs character recognition on the character area within the processing area using the second library stored in the second library storage unit 360, and recognizes the character string of the recognized characters. Therefore, the processing area character recognition unit 810 notifies the determination unit 820 of the candidate "label" corresponding to the candidate of the device data.

[0147] The determination unit 820 is an example of an output unit, and acquires the candidate of the "label" corresponding to the candidate of the device data notified from the processing area character recognition unit 810. In addition, the determination unit 820 refers to the label list held in the label list holding unit 370, and determines whether the character string recognized in the processing area matches any label included in the label list. Thus, the "label" corresponding to the candidate of the device data is determined in the determination unit 820.

[0148] In addition, when candidates with the same "label" are obtained from different regions due to partial overlap of different processing regions, the determination unit 820 determines the more appropriate device data candidate as the device data corresponding to the corresponding label. The so-called more appropriate device data candidate refers to, for example, the device data obtained by:

[0149] The distance between a device data candidate and a “label” candidate, and

[0150] The distance between the candidate of the other device data and the candidate of the "label"

[0151] A candidate for machine data that is closer to one party when compared.

[0152] Alternatively, the more appropriate device data candidates may be, for example, those obtained by:

[0153] ·The row to which a candidate for device data belongs and the candidate for "label", and

[0154] · The row to which the candidate of the other device data belongs to the candidate of "label"

[0155] Candidates for device data that belong to the same row or a row that is closer when comparing.

[0156] Furthermore, the determination unit 820 outputs the candidate of the device data corresponding to the identified label from among the candidates of the device data notified from the extraction unit 310 as the device data together with the identified label.

[0157] <Specific Example of Processing by the Identification Unit>

[0158] Next, a specific example of the processing of the recognition unit 330 will be described. Fig. 9 It is a diagram showing a specific example of the processing of the recognition unit.

[0159] in, Fig. 9 9a shows a situation where the processing area character recognition unit 810 performs character recognition on the character area in the processing area 710 determined based on the information of the boundary box 701. Fig. 9 9a, as a label corresponding to the candidate of device data, determines "model" (model name) (symbol 921), and outputs "model: A12BCD".

[0160] in addition, Fig. 9 9b shows a situation where the processing area character recognition unit 810 performs character recognition on the character area in the processing area 911 determined based on the information of the boundary box 702. Fig. 9 9b, as a label corresponding to the candidate of device data, determines "manufacturing number" (symbol 922), and outputs "manufacturing number: EF34567".

[0161] in addition, Fig. 9 9c shows a situation where the processing area character recognition unit 810 performs character recognition on the character area in the processing area 912 determined based on the information of the boundary box 703. Fig. 9 9c, as a label corresponding to the candidate of the equipment data, "compressor motor output" (compressor capacity) (symbol 923) is determined, and "compressor motor output: yy.y kW" is output.

[0162] in addition, Fig. 9 9d shows that in the processing area 913 determined based on the information of the boundary box 704, the processing area character recognition unit 810 performs character recognition on the area of ​​characters. Fig. 9 In the case of 9d, the label corresponding to the candidate of the device data is not determined and the device data is not output.

[0163] More specifically, in Fig. 9In 9d, "compressor motor output" recognized by characters in processing area 913 becomes a candidate for a label, but "compressor motor output" is also recognized by characters in processing area 912 and becomes a candidate for a label. In addition, the label candidates and the device data candidates in processing area 912 are located in the same row, while the label candidates and the device data candidates in processing area 913 are located in different rows. Therefore, "compressor motor output" is determined as a label corresponding to the device data candidate in processing area 912. Therefore, the judgment unit 820 determines that a label corresponding to the device data candidate cannot be determined in processing area 913. As a result, the device data is not output from processing area 913.

[0164] in addition, Fig. 9 9e shows a situation where the processing area character recognition unit 810 performs character recognition on the character area in the processing area 914 determined based on the information of the boundary box 705. Fig. 9 9e, determine the label corresponding to the candidate of the equipment data (symbol 924), and output "refrigerant amount: zz.z kg".

[0165] in addition, Fig. 9 9f shows a situation where the processing area character recognition unit 810 performs character recognition on the character area in the processing area 915 determined based on the information of the boundary box 706. Fig. 9 9f, as a label corresponding to the candidate of the device data, determines "refrigerant" (symbol 924), and outputs "refrigerant type: G89".

[0166] in addition, Fig. 9 9g shows a situation where the processing area character recognition unit 810 performs character recognition on the character area in the processing area 720 determined based on the information of the boundary box 707. Fig. 9 In 9g, a label corresponding to a candidate for device data is not specified, and the device data is not output.

[0167] More specifically, in Fig. 9 In 9g, since “LOW PRESSURE SECTION” recognized by the characters in the processing area 720 is a character string not included in the label list, the determination unit 820 determines that the label corresponding to the candidate of the device data cannot be determined.

[0168] Thus, the recognition unit 330

[0169] Determine the processing area based on the bounding box corresponding to the candidate in the device data,

[0170] Determine the label by performing character recognition on the character area of ​​the determined processing area,

[0171] Candidates of device data corresponding to the identified label are output as device data in association with the identified label.

[0172] As a result, the identification unit 330 outputs the following as the device data read from the device nameplate 111:

[0173] Model: A12BCD,

[0174] Manufacturing number: EF34567,

[0175] Compressor motor output: yy.y kW,

[0176] ·Refrigerant quantity: zz.z kg,

[0177] Refrigerant type: G89.

[0178] <Flow of Character Recognition Processing by Character Recognition Device>

[0179] Next, the flow of the character recognition process by the character recognition device 120 will be described. Fig.10 as well as Fig.11 These are the first and second flowcharts showing the flow of the character recognition process.

[0180] In step S1001 , the extraction unit 310 acquires a captured image generated by capturing the device nameplate 111 with the imaging device 205 .

[0181] In step S1002 , the extraction unit 310 extracts a region of characters included in the acquired captured image.

[0182] In step S1003 , the extraction unit 310 performs character recognition on the extracted character region using the first library.

[0183] In step S1004 , the extraction unit 310 recognizes a character string from the recognized characters, and specifies a character string including the characters specified by the device data rule among the recognized character strings.

[0184] In step S1004 , if it is determined that the character string does not include characters defined by the device data rule (in the case of “No” in step S1004 ), the process proceeds to step S1006 .

[0185] On the other hand, in step S1004 , if it is determined that the string is a string including characters defined by the device data rule (if “YES” in step S1004 ), the process proceeds to step S1005 .

[0186] In step S1005, the extraction unit 310 determines the recognized character string as a candidate for device data.

[0187] In step S1006, the extraction section 310 determines whether character recognition has been performed on the entire region of the characters extracted in step S1002.

[0188] In step S1006 , when it is determined that there is an area where characters that have not been subjected to character recognition exist (in the case of “NO” in step S1006 ), the process returns to step S1003 .

[0189] On the other hand, in step S1006 , when it is determined that character recognition has been performed on the entire region of the characters extracted in step S1002 (in the case of “YES” in step S1006 ), the process proceeds to step S1007 .

[0190] In step S1007 , the decision unit 320 decides a processing region based on the bounding box used when extracting candidates for device data.

[0191] exist Fig.11 In step S1101 , the recognition unit 330 extracts a character region from the determined processing region.

[0192] In step S1102 , the recognition unit 330 performs character recognition on the extracted character region using the second library.

[0193] In step S1103 , the recognition section 330 refers to the label list to determine whether the character string recognized in the processing area corresponds to any label included in the label list.

[0194] In step S1103 , if it is determined that the character string recognized in the processing area does not correspond to any label included in the label list (in the case of “No” in step S1103 ), the process proceeds to step S1105 .

[0195] On the other hand, in step S1103 , if it is determined that the character string recognized in the processing area matches any label included in the label list (if “YES” in step S1103 ), the process proceeds to step S1104 .

[0196] In step S1104, the identification unit 330 determines the candidate of the device data as the device data corresponding to the corresponding tag.

[0197] In step S1105 , the recognition unit 330 determines whether character recognition has been performed on all regions of the characters extracted in step S1101 .

[0198] In step S1105 , when it is determined that there is an area where characters that have not been subjected to character recognition exist (in the case of NO in step S1105 ), the process returns to step S1103 .

[0199] On the other hand, in step S1105 , when it is determined that character recognition has been performed on the entire region of the characters extracted in step S1101 (in the case of YES in step S1105 ), the process proceeds to step S1106 .

[0200] In step S1106, the recognition unit 330 determines whether there are multiple device data candidates on the same tag. In step S1106, if it is determined that there are no multiple device data candidates (in the case of "No" in step S1106), the process proceeds to step S1108.

[0201] On the other hand, in step S1106 , when it is determined that there are a plurality of candidates for device data (in the case of YES in step S1106 ), the process proceeds to step S1107 .

[0202] In step S1107, the identification unit 330 determines a more appropriate device data candidate from among the plurality of device data candidates.

[0203] In step S1107 , the recognition unit 330 associates the determined device data candidate with the corresponding label and outputs it as device data corresponding to the corresponding label, and then ends the character recognition process.

[0204] <Summary>

[0205] As can be seen from the above description, according to the character recognition device 120 of the first embodiment,

[0206] Extracting device data candidates from the captured image based on the first library.

[0207] The processing area of ​​the captured image is determined based on the position of the bounding box used when extracting device data candidates.

[0208] · Perform character recognition on the processing area determined in the captured image.

[0209] Thus, according to the first embodiment, by performing character recognition processing on an image captured by capturing a device nameplate, it is possible to shorten the reading time when reading device data.

[0210] [Second embodiment]

[0211] In the first embodiment described above, the determination unit 320 determines the processing area based on the bounding box when the extraction unit 310 extracts the device data candidates. However, the determination unit 320 may determine the processing area based on information other than the bounding box (the height and width of the character string that is the device data candidate, information indicating the position in the captured image, etc.).

[0212] In addition, in the first embodiment described above, the decision unit 320 calculates the height information of the processing area by setting the height information of the bounding box to a predetermined multiple, and calculates the width information of the processing area by expanding the bounding box to the end in the width direction. However, the method of calculating the height information and the width information when the decision unit 320 determines the processing area is not limited to this. For example, in the width direction, the width information of the processing area may also be calculated by setting the width information of the bounding box to a predetermined multiple. Alternatively, the width information of the processing area may be calculated by, for example, expanding to the end of the image area 122 of the device nameplate 111 instead of the end in the width direction of the captured image.

[0213] In addition, in the first embodiment, when it is determined that there are multiple device data candidates in the same label, the more appropriate device data candidate is determined by comparing the distance between the label and the device data candidate. Alternatively, the more appropriate device data candidate is determined by comparing the rows to which the label and the device data candidate belong. However, the method of determining the more appropriate device data candidate is not limited to these. For example, a combination rule of the device data candidate and the label may be prepared in advance, and the determination may be made based on the combination rule.

[0214] In the first embodiment, the case where the device data is read from the device nameplate is described, but the object of the device data is not limited to the device nameplate. Alternatively, the object read from the device nameplate is not limited to the device data. Furthermore, the object photographed by the camera is not limited to the device nameplate, as long as it contains a specific character string.

[0215] In addition, in the first embodiment, the device data rule is exemplified when the field device is an outdoor unit of an air conditioner, but when the field device is different, the device data rule corresponding to the different field device is used. Similarly, in the first embodiment, the tag list is exemplified when the field device is an outdoor unit of an air conditioner, but when the field device is different, the tag list corresponding to the different field device is used.

[0216] Although the embodiments have been described above, it will be understood that various changes in form and details may be made without departing from the spirit and scope of the claims.

[0217] This application claims priority based on Japanese Patent Application No. 2021-109954 filed on July 1, 2021, the entire contents of which are incorporated into this application by reference.

[0218]

Explanation of symbols

[0219] 110: Field equipment

[0220] 111: Equipment nameplate

[0221] 120: Character recognition device

[0222] 121: Capture image

[0223] 205: Camera

[0224] 310: Extraction Department

[0225] 320: Decision Department

[0226] 330: Identification Department

[0227] 410: String recognition unit

[0228] 420: Specific character string determination unit

[0229] 430: Bounding box determination unit

[0230] 440: Device Data Rules

[0231] 610: Height information calculation unit

[0232] 620: Width information calculation unit

[0233] 630: Processing area determination unit

[0234] 701-707: Bounding box

[0235] 710, 720: Processing area

[0236] 810: Processing area character recognition unit

[0237] 820: Judgment Department

[0238] 911~915: Processing area.

Claims

1. A character recognition device, comprising: an extraction unit that extracts a specific character string from the captured image, the specific character string including: a character string obtained by recognizing the character string using alphanumeric characters constituting the first library and a character structure specified by the determination information; a determination unit that determines a processing area of ​​the captured image based on a position of the extracted specific character string so as to include a label corresponding to the extracted specific character string; and The recognition unit performs character recognition on a processing area determined in the captured image using a second library having more character types than the first library.

2. The character recognition device according to claim 1, wherein: A first storage unit is provided to store the judgment information. The extraction unit performs character recognition on the captured image using the first library, and determines a character string including characters specified by the determination information from among the recognized character recognition character strings, thereby extracting the specific character string.

3. The character recognition device according to claim 2, wherein: The determination information specifies any one of the number of characters, the type of characters, and the order of characters as the composition of the characters.

4. The character recognition device according to claim 3, wherein: The determination unit determines as the processing area an area that includes a bounding box used by the extraction unit when extracting the specific character string, and an area that has a height that is a specified multiple of the height of the bounding box and has a width from the position of the specific character string to the end position of the captured image, so as to include a label corresponding to the extracted specific character string.

5. The character recognition device according to claim 1, wherein: A second storage unit for storing a list of identification information is provided. The recognition unit determines whether the recognized character matches any label included in the recognition information list.

6. The character recognition device according to claim 5, wherein: An output unit is provided for outputting the extracted specific character string as data corresponding to the corresponding label when the recognition unit determines that the character string matches any label included in the identification information list.

7. The character recognition device according to claim 6, wherein: When the plurality of processing areas partially overlap each other and the same label is included in the plurality of processing areas, the output unit outputs a specific character string located closer to the same label as data corresponding to the same label.

8. The character recognition device according to claim 5, wherein: The labels included in the identification information list are labels of equipment, and include at least one of a model name, a manufacturing number, a manufacturing year and month, a refrigerant type, a refrigerant amount, and a compressor capacity.

9. The character recognition device according to claim 1, wherein: The captured image is an image obtained by capturing a nameplate attached to the equipment.

10. A character recognition method comprising: an extraction step of extracting a specific character string from the captured image, the specific character string comprising: a character string obtained by recognizing the character string using alphanumeric characters constituting the first library and a character structure specified by the determination information; a determining step of determining a processing area of ​​the captured image based on a position of the extracted specific character string so as to include a label corresponding to the extracted specific character string; and The recognition step performs character recognition on the processing area determined in the captured image using a second library having more character types than the first library.

11. A character recognition program product, the character recognition program product comprising a computer program, characterized in that: When the computer program is executed by a processor, it causes the computer to execute: an extraction step of extracting a specific character string from the captured image, the specific character string comprising: a character string obtained by recognizing the character string using alphanumeric characters constituting the first library and a character structure specified by the determination information; a determining step of determining a processing area of ​​the captured image based on a position of the extracted specific character string so as to include a label corresponding to the extracted specific character string; as well as The recognition step performs character recognition on the processing area determined in the captured image using a second library having more character types than the first library.

Images