Information terminal, character recognition method, and character recognition program product

Abstract

To reduce the work load of a worker when acquiring device data of a site device by shooting. An information terminal has a control section that repeatedly performs a process of acquiring a shot frame image and a process of storing a character recognition result obtained by performing character recognition on the frame image in a storage section, and determines the character recognition result when a prescribed operation of the information terminal is accepted.

Description

TECHNICAL FIELD

[0001] The present application relates to an information terminal, a character recognition method, and a character recognition program product. BACKGROUND

[0002] A technique is known in which device data of a field device such as an air conditioning device is acquired by capturing a device plate attached to the field device using an information terminal or the like and performing character recognition processing. According to this technique, for example, a maintenance worker is able to manage device data of a field device on which a field survey has been performed, by simply capturing a device plate.

[0003] Patent Literature 1: Japanese Patent Application Publication No. 2020-091748 SUMMARY

[0004] <Problems to be Solved by the Invention>

[0005] On the other hand, when capturing is performed at a field, depending on the capturing situation, character recognition processing can be performed inappropriately, erroneous recognition can occur, and as a result, correct device data can not be acquired for part or all of the device data. In this case, the maintenance worker needs to perform capturing again, and the work load is high for the maintenance worker.

[0006] The present application can reduce the work load of a worker when device data of a field device is acquired by capturing.

[0007] <Means for Solving the Problems>

[0008] A first aspect of the present application is an information terminal having a control section, wherein

[0009] the control section:

[0010] repeatedly performs processing of acquiring a captured frame image and processing of storing a character recognition result obtained by performing character recognition on the frame image in a storage section,

[0011] determines the character recognition result in a case where a prescribed operation on the information terminal is accepted.

[0012] According to the first aspect of the present application, it is possible to reduce the work load of a worker when device data of a field device is acquired by capturing.

[0013] Further, a second aspect of the present application is the information terminal according to the first aspect, wherein

[0014] the character recognition result obtained by performing the character recognition is displayed overlaid when the captured frame image is displayed in real time.

[0015] Further, a third aspect of the present application is the information terminal according to the second aspect, in which

[0016] The control section:

[0017] The frame image displayed is updated at a first time interval,

[0018] The text recognition result displayed is updated at a second time interval longer than the first time interval.

[0019] Further, a fourth aspect of the present application is the information terminal according to any one of the first to third aspects, in which

[0020] The control section:

[0021] In a case where the photographed frame image contains a plurality of items to be stored as text recognition results, the text recognition result of each item obtained by performing the text recognition is stored in the storage section in units of items,

[0022] The text recognition result is determined in units of items in a case where a prescribed operation on the information terminal is accepted.

[0023] Further, a fifth aspect of the present application is the information terminal according to any one of the first to fourth aspects, in which

[0024] The control section:

[0025] In a case where the displayed text recognition result is corrected in correspondence with the display of the determined text recognition result on the display section, the corrected text recognition result is stored in the storage section.

[0026] Further, a sixth aspect of the present application is the information terminal according to any one of the first to fifth aspects, in which

[0027] The control section:

[0028] A model name and a production number are obtained as text recognition results by performing text recognition on a frame image in which a nameplate of an apparatus is photographed.

[0029] Further, a seventh aspect of the present application is the information terminal according to any one of the first to sixth aspects, in which

[0030] The control section:

[0031] The text recognition result obtained by performing the text recognition is stored in a first storage section,

[0032] The character recognition result determined by accepting the prescribed operation on the information terminal is stored in the second storage section.

[0033] Further, an eighth aspect of the present application is the information terminal according to the seventh aspect, in which

[0034] The control section:

[0035] The character recognition result stored in the first storage section is updated each time a character recognition result is obtained by performing the character recognition.

[0036] Further, a ninth aspect of the present application is the information terminal according to any one of the first to eighth aspects, in which

[0037] The character recognition result is a result among results obtained by performing the character recognition, which is determined by the control section to satisfy a saving condition.

[0038] Further, a tenth aspect of the present application is the information terminal according to the ninth aspect, in which

[0039] The control section:

[0040] In a case where a result obtained by performing the character recognition for a prescribed item is a character string set in advance, it is determined that the saving condition is satisfied, and the result obtained for the prescribed item is stored in the storage section as a character recognition result for the prescribed item.

[0041] Further, an eleventh aspect of the present application is the information terminal according to the seventh aspect, in which

[0042] The control section:

[0043] A frame image for which the character recognition result stored in the first storage section is obtained is stored in the first storage section in association with the character recognition result,

[0044] A frame image for which the character recognition result stored in the second storage section is obtained is stored in the second storage section in association with the character recognition result.

[0045] Further, a twelfth aspect of the present application is a character recognition method in which the following processes are performed by a control section of an information terminal:

[0046] The process of obtaining a captured frame image and the process of storing a character recognition result obtained by performing character recognition on the frame image in a storage section are repeatedly performed; and

[0047] In a case where a prescribed operation on the information terminal is accepted, a character recognition result is determined.

[0048] According to the 12th aspect of the present application, it is possible to reduce the work load of a worker when acquiring device data of a site device by imaging.

[0049] Further, the 13th aspect of the present application is a character recognition program for causing a control section of an information terminal to execute the following processes:

[0050] the process of obtaining a character recognition result by performing character recognition on the frame image, and the process of storing the character recognition result in a storage section; and

[0051] In a case where a prescribed operation to the information terminal is accepted, the character recognition result is determined.

[0052] According to the 13th aspect of the present application, it is possible to reduce the work load of a worker when acquiring device data of a site device by imaging. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a diagram showing a usage example of an information terminal.

[0054] Figure 2 is a diagram showing an example of a character recognition result obtained by performing a character recognition process on each frame image.

[0055] Figure 3 is a diagram showing an example of a hardware structure of an information terminal.

[0056] Figure 4 is a first diagram showing an example of a functional structure of an information terminal.

[0057] Figure 5 is a diagram showing an example of an action of an information terminal as a whole.

[0058] Figure 6 is a diagram showing a specific example of a process from the recognition section to the extraction section.

[0059] Figure 7 is a diagram showing a specific example of a process of the determination section.

[0060] Figure 8 is a flowchart showing a processing flow of the first storage control section.

[0061] Figure 9 is a flowchart showing a processing flow of the second storage control section.

[0062] Figure 10A is a first diagram showing a display example of a frame image and a frame by a display section.

[0063] Figure 10BFig. 2 is a display example of a frame image and a frame displayed by the display section.

[0064] Figure 11 Fig. 4 is a flowchart showing a flow of character recognition processing performed by the information terminal.

[0065] Figure 12 Fig. 2 is a functional configuration example of the information terminal. DETAILED DESCRIPTION

[0066] Hereinafter, each embodiment will be described with reference to the drawings. Note that in the present specification and the drawings, constituent elements having substantially the same function structure are given the same reference numeral, and repetitive description is omitted.

[0067] [First Embodiment]

[0068] <Usage Example of Information Terminal>

[0069] First, a usage example of the information terminal of the first embodiment will be described. Figure 1 Fig. 1 is a diagram showing the usage example of the information terminal. As shown in the left side of Fig. 1, the information terminal 120 is used in a scene where, for example, device data is read from a device nameplate 111 attached to a side surface or the like of a field device 110. Figure 1

[0070] According to the information terminal 120, for example, when the maintenance worker 130 performs a field investigation and registers the investigation result in a server device (not shown) or the like, the maintenance worker 130 can be relieved of the trouble of manually inputting device data for specifying the field device 110 as an investigation target. Further, the maintenance worker 130 can be prevented from erroneously inputting the device data.

[0071] Note that in the present embodiment, it is assumed that the field device 110 is an outdoor unit of an air conditioning device, and the information terminal 120 is a portable terminal. However, the field device 110 is not limited to the outdoor unit of the air conditioning device, but can be a unit other than the outdoor unit of the air conditioning device, and can be a field device other than the air conditioning device. Further, the information terminal 120 is not limited to the portable terminal, but can be an information terminal other than the portable terminal, as long as it is a terminal having an optical character recognition function.

[0072] Figure 1 The right side of Fig. 1 shows a case where the maintenance worker 130 has taken an image of the device nameplate 111 and displays the taken image 121 on the display screen of the information terminal 120. As shown in the right side of Fig. 1, the taken image 121 includes an image region 122 corresponding to the device nameplate 111. Figure 1

[0073] ​​In a case where the field device 110 is an outdoor unit of an air conditioning device, in the device nameplate 111 displayed in the image area 122, for example, the following are included:

[0074] • a description indicating that it is an outdoor unit;

[0075] • a description indicating that it is an outdoor unit for both cooling and heating;

[0076] • a description about the model (nameplate, model code data);

[0077] • a description about the production number (nameplate, production number data);

[0078] • a description about the product quality (nameplate, quality data);

[0079] • a description about the power supply specifications (nameplate, number of phases data, voltage data, frequency data);

[0080] • a description about the output of the compressor motor (nameplate, power consumption data);

[0081] • a description about the management number (nameplate, management code data);

[0082] • a description about the amount and type of refrigerant (nameplate, refrigerant amount data, refrigerant type code);

[0083] • a description about the design pressure (nameplate, pressure data);

[0084] • a description about the production date (nameplate, production date data);

[0085] and the like (in Figure 1 , the description indicated by “*” is omitted). Furthermore, the actual description of the device nameplate 111 includes many types of characters, such as letters, numbers, symbols, and Chinese characters (language of the country where the field device 110 is installed).

[0086] Note that, in the present embodiment, when the field device 110 is photographed by the information terminal 120, the photographing is performed as dynamic image data during a period from when the maintenance worker 130 starts the photographing to when the shutter operation is performed. Furthermore, in the present embodiment, character recognition processing is performed on each frame image included in the dynamic image data.

[0087] <Example of Character Recognition Processing>

[0088] Next, the character recognition result when the maintenance worker 130 photographs the field device 110 as dynamic image data and the character recognition processing is performed on each frame image by the information terminal 120 will be described. Figure 2is an example of a character recognition result obtained by performing character recognition processing on each of the frame images.

[0089] In Figure 2 In the example of the time axis 201, Figure 2 The example of the time axis 201 indicates frame images (symbols 211 to 213) photographed at times = T1, T2, T3 among the frame images included in the moving image data.

[0090] Further, Figure 2 The example of the time axis 201 indicates that, by performing character recognition processing on each of the frame images included in the moving image data, character recognition results on the model name, the production number, and the production date of the field device 110 are acquired as the device data (symbols 221 to 223).

[0091] Specifically, Figure 2 The example of the time axis 201 indicates that, in the case of the frame image 1 photographed at time = T1, character recognition results are obtained by performing character recognition processing on the model name and the production number. On the other hand, Figure 2 The example of the time axis 201 indicates that, by performing character recognition processing on the production date, no character recognition result is obtained.

[0092] Further, Figure 2 The example of the time axis 201 indicates that, in the case of the frame image 2 photographed at time = T2, character recognition results are obtained by performing character recognition processing on any one of the model name, the production number, and the production date, and the device data is acquired.

[0093] Further, Figure 2 The example of the time axis 201 indicates that, in the case of the frame image 3 photographed at time = T3, a character recognition result is obtained by performing character recognition processing on the production number. On the other hand, Figure 2 The example of the time axis 201 indicates that, by performing character recognition processing on the model name and the production date, no character recognition result is obtained.

[0094] As described above, in the case where the field device 110 is photographed as moving image data, there are cases where a character recognition result is obtained and cases where no character recognition result is obtained due to a slight difference in the photographing condition during the photographing that is not noticed by the maintenance worker 130.

[0095] In other words, at the time point when the shutter operation is performed, it is not highly likely that a character recognition result is obtained for all the plurality of object items (the plurality of items to be recognized). Therefore, in the information terminal 120 of the present embodiment, character recognition processing is performed on the frame images photographed before the shutter operation is performed, respectively. Then, in the information terminal 120 of the present embodiment, in the case where a character recognition result is obtained for all the object items in any frame image, the character recognition result is acquired as the device data.

[0096] Thus, as compared with a case where character recognition processing is performed only on a frame image acquired at a time point at which a shutter operation is performed, it is possible to improve the possibility of acquiring device data. Therefore, according to the information terminal 120 of the present embodiment, it is possible to reduce the work load of the maintenance worker when acquiring device data of an on-site device by photographing.

[0097] <Hardware structure of information terminal>

[0098] Next, the hardware structure of the information terminal 120 will be described. Figure 3 is a diagram showing an example of the hardware structure of the information terminal. As shown in Figure 2 , the information terminal 120 includes a processor 301, a memory 302, an auxiliary storage device 303, a user interface device 304, an imaging device 305, and a communication device 306. Note that each hardware of the information terminal 120 is connected to each other via a bus 307.

[0099] The processor 301 includes various computing devices such as a CPU (Central Processing Unit). The processor 301 reads out various programs (e.g., a character recognition program described later, etc.) onto the memory 302 and executes them.

[0100] The memory 302 includes main storage devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processor 301 and the memory 302 form a so-called computer, which realizes various functions by executing various programs read out onto the memory 302 by the processor 301.

[0101] The auxiliary storage device 303 stores various programs and various data (e.g., frame images, libraries, correction tables, device data rules, character recognition results, etc. described later) to be used when the processor 301 executes various programs.

[0102] The user interface device 304 has, for example, an operation function for performing various operations of the maintenance worker 130 and a display function for displaying the processing results of various processes performed by the information terminal 120.

[0103] The imaging device 305 photographs the device nameplate 111 and generates a photographed image 121. The communication device 306 is a communication device for communicating with external devices (e.g., a server device not shown) via a network.

[0104] Note that various programs installed in the auxiliary storage device 303 are installed, for example, by being downloaded from a network via the communication device 306.

[0105] <Function structure of information terminal>

[0106] Next, the function structure of the information terminal 120 is described. Figure 4 FIG. 1 is an example of a function structure of an information terminal. As described above, the information terminal 120 has installed therein a character recognition program, and by executing the program, the information terminal 120 functions as:

[0107] • an operation accepting section 401,

[0108] • an acquisition section 402,

[0109] • a recognition section 403,

[0110] • a correction section 404,

[0111] • an extraction section 405,

[0112] • a determination section 406,

[0113] • a bounding box generation section 407,

[0114] • a first storage control section 408,

[0115] • a second storage control section 409,

[0116] • a display section 410,

[0117] • a manual correction section 411.

[0118] The operation accepting section 401 accepts a photographing start operation and a shutter operation among various operations performed by the maintenance worker 130 on the information terminal 120. Further, in a case where the photographing start operation is accepted, the operation accepting section 401 notifies the acquisition section 402 of a photographing start instruction.

[0119] On the other hand, in a case where the shutter operation is accepted, the operation accepting section 401 notifies the acquisition section 402 of a photographing end instruction, and notifies the second storage control section 409 that the shutter operation has been performed.

[0120] When the photographing start instruction is notified from the operation accepting section 401, the acquiring section 402 causes the imaging device 305 to operate, and starts acquiring photographing image data that is photographed as a dynamic image with a prescribed frame period (an example of a prescribed period). The acquiring section 402 notifies the display section 410 of each frame image included in the acquired photographing image data, and interval extracts each frame image, and notifies the recognizing section 403. Further, the acquiring section 402 stores each frame image notified to the recognizing section 403 in the frame image storage section 421. Note that these processes of the acquiring section 402 are repeatedly executed until the photographing end instruction is notified from the operation accepting section 401.

[0121] The recognizing section 403 performs character recognition processing on each frame image notified by the acquiring section 402 while referring to the library storage section 422, and notifies the corrected section 404 of the recognized character. Note that the library storage section 422 stores a library for performing character recognition on letters, numbers, and symbols.

[0122] The corrected section 404 corrects a character that is erroneously recognized in the character recognition processing among the characters notified by the recognizing section 403 while referring to the correction table storage section 423, and corrects the recognized character to a correct character. Further, the corrected section 404 notifies the extracting section 405 of the corrected character. Note that the correction table storage section 423 stores a correction table in which a character that is easily erroneously recognized when performing character recognition processing on a frame image included in photographing image data obtained by photographing a photographing device nameplate is associated with a character that is correctly recognized.

[0123] The extracting section 405 extracts character strings from the corrected characters notified by the corrected section 404, and notifies the determining section 406 of the extracted character strings together with their respective positions and sizes. Note that the extracting section 405 extracts one line as one character string for the corrected characters. When a blank including a prescribed number of characters is included, one line is determined by line feed. However, in the present embodiment, the extracting section 405 does not line feed when a blank is present after a number.

[0124] The determining section 406 determines whether the character strings notified from the extracting section 405 are described in accordance with the naming rule of the object item (in the present embodiment, an item in which a model name, a production number, and a production date are recognized) (whether the saving condition is satisfied). Note that the naming rule in which the character type, the number of characters, and the arrangement order of the character strings that constitute the object item are prescribed is stored in the device data rule storage section 424, and the determining section 406 refers to the device data rule storage section 424 to determine whether the character strings are described in accordance with the naming rule of the object item.

[0125] Further, if the result of the determination by the determination section 406 is that it is determined that the description is made in accordance with the naming rule of the object item, the

[0126] • the character string (character recognition result) determined to be described in accordance with the naming rule from the character string notified from the extraction section 405; and

[0127] • the position and size of the character string,

[0128] to the frame generation section 407.

[0129] Further, if the result of the determination by the determination section 406 is that it is determined that all the object items are described in accordance with the corresponding naming rule, each character string (each character recognition result) described in accordance with the corresponding naming rule is notified to the 1st storage control section 408 in association with each object item. Also, the determination section 406 notifies the 1st storage control section 408 of the position and size notified from the extraction section 405 in association with each object item.

[0130] The frame generation section 407 generates a frame (information indicating the position and size of the character string) based on the position and size of the character string notified by the determination section 406. Further, the frame generation section 407 notifies the display section 410 of the generated frame together with the character string (character recognition result) notified by the determination section 406.

[0131] When the character recognition result with respect to all the object items is notified by the determination section 406, the 1st storage control section 408 stores the position and size in association with the object item as the device data into the 1st storage section 425. When the character recognition result with respect to all the object items is notified by the determination section 406, the 1st storage control section 408 reads out the corresponding frame image from the frame image storage section 421 and stores into the 1st storage section 425.

[0132] Note that, when the character recognition result with respect to the object item has been stored into the 1st storage section 425 together with the corresponding position and size and frame image, the 1st storage control section 408 performs overwrite to store the character recognition result. Thereby, the latest character recognition result among the character recognition results with respect to all the object items repeatedly notified during the photographing period is stored into the 1st storage section 425 together with the corresponding position and size and frame image.

[0133] When the shutter operation is notified by the operation accepting section 401, the 2nd storage control section 409 reads out the latest character recognition result, the corresponding position and size, and the frame image from the 1st storage section 425, and stores them in the 2nd storage section 426. Thus, the 2nd storage control section 409 can store the determined character recognition result and the like in the 2nd storage section 426. In other words, in the present embodiment, the shutter operation can be considered as an operation for determining the character recognition result and the like.

[0134] Further, the 2nd storage control section 409 notifies the frame generating section 407 of the determined character recognition result, position, and size. Further, the 2nd storage control section 409 notifies the display section 410 of the determined frame image.

[0135] Further, in a case where a corrected character string obtained by manually correcting the character recognition result by the manual correction section 411 in correspondence with the notification of the determined character recognition result, position, and size, and frame image is notified, the 2nd storage control section 409 receives the corrected character string. Then, the 2nd storage control section 409 overwrites and stores the received corrected character string in the 2nd storage section 426.

[0136] The display section 410 displays each frame image notified by the acquisition section 402 at a prescribed frame period on the user interface device 304. Note that the display section 410 overlaps the frame and the character recognition result notified from the frame generating section 407 when displaying each frame image notified at the prescribed frame period. The frame and the character recognition result overlapped on each frame image by the display section 410 are updated at a period longer than the prescribed frame period. This is because, as described above, the acquisition section 402 performs interval extraction on each frame image to notify the recognition section 403, so the period of generating the frame is longer than the prescribed frame period.

[0137] Further, since the shutter operation is performed, the display section 410 overlaps the frame and the character recognition result notified from the frame generating section 407 on the frame image notified from the 2nd storage control section 409. Further, the display section 410 displays the frame image on which the frame and the character recognition result are overlapped on the user interface device 304.

[0138] That is, the display section 410

[0139] • displays the frame image in real time in photographing, and updates the frame and the character recognition result overlapped on the object item at a period longer than the prescribed frame period,

[0140] • when the shutter operation is performed, displays the determined frame image, the frame generated based on the determined position and size, and the determined character recognition result.

[0141] When the manual correction instruction is input in correspondence with the display of the determined character recognition result and the determined frame image by the display section 410 due to the shutter operation, the manual correction section 411 accepts the correction instruction. Further, the manual correction section 411 notifies the second storage control section 409 of the manually corrected corrected character string.

[0142] <Example of the operation of the information terminal as a whole>

[0143] Next, an example of the operation of the information terminal 120 as a whole will be described. Figure 5 is a view showing an example of the operation of the information terminal as a whole.

[0144] In Figure 5 , "frame image 1" to "frame image 5" show the frame images acquired by the acquisition section 402. Specifically, it shows the case where "frame image 1" is acquired by inputting a shooting start instruction, and the shutter operation is performed immediately after "frame image 5" is shot.

[0145] Further, Figure 5 the example of shows the case where "frame image 1", "frame image 3", and "frame image 5" are stored in the frame image storage section 421.

[0146] Note that, according to the example of Figure 5 , the frame image displayed in the user interface device 304 of the information terminal 120 is updated at the first time interval a. Further, the frame image and the character recognition result displayed in the user interface device 304 of the information terminal 120 are updated at least at the second time interval b (a < b).

[0147] Further, in Figure 5 , "recognition result 1 (NG)" shows the case where the recognition section 403 to the determination section 406 each perform processing on "frame image 1" and determine that any one of the character strings included in "frame image 1" is not described in accordance with the naming rule of the object item. At this time, "recognition result 1 (NG)" and "frame image 1" are not stored in the first storage section 425.

[0148] On the other hand, in Figure 5 , "recognition result 3 (OK)" shows the case where the recognition section 403 to the determination section 406 each perform processing on "frame image 3" and determine that the character string included in "frame image 3" is described in accordance with the naming rule of the object item. At this time, "recognition result 3 (OK)" (the character recognition result) and "frame image 3" (note that, in the example of Figure 5 , the position and the size are omitted) are stored in the first storage section 425.

[0149] Further, in Figure 5In the case of "recognition result 5 (OK)", the recognition section 403 to the determination section 406 have executed processing on "frame image 5" and determined that the character string included in "frame image 5" is described in accordance with the naming rule of the object item. At this time, "recognition result 5 (OK)" (character recognition result) and "frame image 5" are overwritten and stored in the 1st storage section 425.

[0150] Further, Figure 5 The example of "recognition result 5 (OK)" (character recognition result) and "frame image 5" stored in the 1st storage section 425 at the time point when the shutter operation is performed are stored in the 2nd storage section 426.

[0151] <Details of processing of each functional section of the information terminal>

[0152] Next, details of processing of each functional section (here, the recognition section 403, the correction section 404, the extraction section 405, the determination section 406, the 1st storage control section 408, and the 2nd storage control section 409) of the information terminal will be described.

[0153] (1) Details of processing from the recognition section 403 to the extraction section 405

[0154] First, details of processing from the recognition section 403 to the extraction section 405 will be described. Figure 6 is a diagram showing a specific example of processing from the recognition section to the extraction section.

[0155] As described above, the library storage section 422 stores a library for character recognition of letters, numerals, and symbols. Therefore, when performing character recognition processing on a frame image, in the case where letters, numerals, and symbols are included within the frame image, the recognition section 403 performs character recognition on the letters, numerals, and symbols within the frame image. Further, in the case where a character other than letters, numerals, and symbols is included within the frame image, the recognition section 403 recognizes the character as any one of letters, numerals, and symbols. Specifically, in the case where a character other than letters, numerals, and symbols is included within the frame image, the recognition section 403 recognizes the character as a letter, numeral, or symbol similar to the character.

[0156] In Figure 6 In the case of "recognition result 5 (OK)", the recognition section 403 to the determination section 406 have executed processing on "frame image 5" and determined that the character string included in "frame image 5" is described in accordance with the naming rule of the object item. At this time, "recognition result 5 (OK)" (character recognition result) and "frame image 5" are overwritten and stored in the 1st storage section 425.

[0157] Subsequently, the correction section 404 determines a character that is erroneously recognized among the characters shown by the reference numeral 610. Specifically, in the case where letters, numerals, and symbols that are not used as object items are included in the characters shown by the reference numeral 610, the correction section 404 determines that these letters, numerals, and symbols are erroneously recognized and performs correction.

[0158] As described above, the correction section 404 refers to the correction table (an example of a character correction rule. For example, the correction table 600) stored in the correction table storage section 423, and corrects the letters, numbers, and symbols that are determined to be misrecognized. Note that, as described above, in the correction table 600, the character that is likely to be misrecognized (the character before correction) is associated in advance with the character (the character after correction) that is correctly recognized when the character is recognized. Referring to Figure 6 As illustrated, the reference numeral 620 indicates the character after correction of the character that is misrecognized in the character illustrated by the reference numeral 610 (note that the reference numeral 620 indicates a state in which, due to the paper surface, further line breaks are performed in order to extract one line as one character string).

[0159] Subsequently, the extraction section 405 extracts one line of the character illustrated by the reference numeral 620 as one character string in order, and notifies the determination section 406. Figure 6 In the example, the reference numeral 630 indicates a case in which "ARR63UP" is extracted as a character string from the character illustrated by the reference numeral 620.

[0160] Further, the extraction section 405 calculates the position and size of the extracted character string in the frame image before the line break in order, and notifies the determination section 406 together with the extracted character string (the reference numeral 630). Figure 6 In the example, the reference numeral 631 indicates a case in which the position and size of the extracted character string (the reference numeral 630) in the frame image before the line break are calculated.

[0161] (2) Details of the processing of the determination section 406

[0162] Next, details of the processing of the determination section 406 will be described. Figure 7 is a diagram that indicates a specific example of the processing of the determination section.

[0163] As described above, the naming rule 700 of the object item is stored in the device data rule storage section 424. By referring to the naming rule 700 stored in the device data rule storage section 424, the determination section 406 determines whether the character string notified from the extraction section 405 in order (refer to the reference numeral 630) is described in accordance with the naming rule of the object item.

[0164] Figure 7 The example of the reference numeral 631 indicates a case in which the character string indicated by the reference numeral 630 is notified, and the determination section 406 refers to the determination criterion set based on the naming rule of "model code data" within the naming rule 700. The determination section 406 determines that the character string illustrated by the reference numeral 630, that is, "ARR63UP", coincides with the determination criterion, and thus is described in accordance with the naming rule of "model code data".

[0165] Accordingly, the determination section 406 notifies the frame generation section 407 of the character string (character recognition result) of the "model name" and its position and size (refer to reference numerals 710 and 711). Figure 7

[0166] Note that, in the example of Figure 7 the character string notified to the determination section 406 in order,

[0167] there is

[0168] • the character string coinciding with the determination reference set based on the naming rule of the "model code data" within the naming rule 700,

[0169] but there is not

[0170] • the character string coinciding with the determination reference set based on the naming rule of the "production number data" within the naming rule 700, and

[0171] • the character string coinciding with the determination reference set based on the naming rule of the "production date data" within the naming rule 700.

[0172] Accordingly, the determination section 406 notifies the first storage control section 408 of the recognition result (NG).

[0173] On the other hand, in the example of Figure 7 the character string notified to the determination section 406 in order,

[0174] there are

[0175] • the character string coinciding with the determination reference set based on the naming rule of the "model code data" within the naming rule 700,

[0176] • the character string coinciding with the determination reference set based on the naming rule of the "production number data" within the naming rule 700, and

[0177] • the character string coinciding with the determination reference set based on the naming rule of the "production date data" within the naming rule 700.

[0178] In this case, the determination section 406 notifies the frame generation section 407 of

[0179] • the character recognition result (recognition result (OK)) regarding all the object items,

[0180] • the position and size of each of the character string (character recognition result) of the "model name", the character string (character recognition result) of the "production number", and the character string (character recognition result) of the "production date"

[0181] ​The first storage control section 408 is notified.

[0182] (3) Details of the processing of the first storage control section 408

[0183] Next, details of the processing of the first storage control section 408 will be described. Figure 8 is a flowchart showing the processing flow of the first storage control section.

[0184] As shown in Figure 8 , in step S801, the first storage control section 408 acquires the character recognition result (recognition result (OK)), position and size, or the recognition result (NG) from the determination section 406.

[0185] In step S802, the first storage control section 408 refers to the frame image storage section 421 and acquires the frame image corresponding to the acquired character recognition result.

[0186] In step S803, the first storage control section 408, in a case where the character recognition result (recognition result (OK)), position and size are acquired, associates the acquired character recognition result with the object item and stores it as the device data into the first storage section 425. Further, the first storage control section 408 also stores the acquired position and size in association with the object item into the first storage section 425. Further, the first storage control section 408 also stores the acquired frame image in association with the acquired character recognition result into the first storage section 425.

[0187] Note that, in a case where the character recognition result, position and size, frame image are already stored in the first storage section 425, the first storage control section 408 overwrites them to store.

[0188] (4) Details of the processing of the second storage control section 409

[0189] Next, details of the processing of the second storage control section 409 will be described. Figure 9 is a flowchart showing the processing flow of the second storage control section.

[0190] As shown in Figure 9 , in step S901, the second storage control section 409 determines the latest character recognition result stored as the device data into the first storage section 425.

[0191] In step S902, the second storage control section 409 reads out the determined character recognition result, the corresponding position and size, and the corresponding frame image from the first storage section 425 and stores them into the second storage section 426, thereby determining the character recognition result and the like.

[0192] In step S903, the 2nd storage control section 409 notifies the display section 410 of the determined frame image stored in the 2nd storage section 426.

[0193] In step S904, the 2nd storage control section 409 notifies the frame generation section 407 of the determined character recognition result, position, and size stored in the 2nd storage section 426.

[0194] <Display example of display section>

[0195] Next, a display example of the frame image and the frame displayed in the display section 410 will be described. Figure 10A and Figure 10B are the first and second diagrams showing a display example of the frame image and the frame displayed by the display section.

[0196] Figure 10A The example of shows a case where the frame image 1 shown by the reference numeral 1011 at time = Tl is displayed when the maintenance worker 130 performs the photographing start operation. Further, Figure 10A The example of shows a case where, in the character recognition processing of the frame image 1, the character recognition result is obtained for the "model name" and the "production number", but the character recognition result is not obtained for the "production date" (see the reference numeral 1021). Therefore, in the frame image 1, the frame is displayed at the character string of the "model name" and the "production number", and the character recognition result is displayed.

[0197] Further, Figure 10A The example of shows a case where the frame image 2 shown by the reference numeral 1012 at time = T2 is displayed. Since the character recognition processing is not performed on the frame image 2, the character recognition result obtained by the character recognition processing on the frame image 1 is displayed as it is. That is, in the frame image 2, the frame is displayed at the character string of the "model name" and the "production number", and the character recognition result is displayed.

[0198] Further, Figure 10A The example of shows a case where the frame image 3 shown by the reference numeral 1013 at time = T3 is displayed. It shows that, in the character recognition processing on the frame image 3, the character recognition result is obtained for the "model name", the "production number", and the "production date" (see the reference numeral 1023). Therefore, in the frame image 3, the frame is displayed at the character string of the "model name", the "production number", and the "production date", and the character recognition result is displayed.

[0199] Next, Figure 10BThe example of time = T4 shows the case where the frame image 4 shown by the reference numeral 1014 is displayed. Since no character recognition processing is performed on the frame image 4, the character recognition result acquired by performing the character recognition processing on the frame image 3 is displayed as it is. That is, in the frame image 4, a frame is displayed at the character string of "model name", "production number", and "production date", and the character recognition result is displayed.

[0200] Further, Figure 10B The example of time = T5 shows the case where the frame image 5 shown by the reference numeral 1015 is displayed. It is shown that in the character recognition processing on the frame image 5, the character recognition result is acquired for "model name", "production number", and "production date" (refer to the reference numeral 1025). Therefore, in the frame image 5, a frame is displayed at the character string of "model name", "production number", and "production date", and the character recognition result is displayed.

[0201] Further, Figure 10B The example of time = T5 shows the case where the frame image 5 shown by the reference numeral 1015 is displayed. It is shown that in the character recognition processing on the frame image 5, the character recognition result is acquired for "model name", "production number", and "production date" (refer to the reference numeral 1025). Therefore, in the frame image 5, a frame is displayed at the character string of "model name", "production number", and "production date", and the character recognition result is displayed.

[0202] < Character recognition processing >

[0203] Next, the flow of the character recognition processing of the information terminal 120 as a whole will be described. Figure 11 is a flowchart showing the flow of the character recognition processing of the information terminal.

[0204] In step S1101, the information terminal 120 accepts the photographing start operation by the maintenance worker 130.

[0205] In step S1102, the information terminal 120 starts photographing of the moving image data, acquires frame images, and displays the acquired frame images.

[0206] In step S1103, the information terminal 120 stores the acquired frame images in the frame image storage section 421 after interval extraction.

[0207] In step S1104, the information terminal 120 performs the character recognition processing on the acquired frame images after interval extraction.

[0208] In step S1105, the information terminal 120 corrects the character recognized by performing the character recognition processing.

[0209] In step S1106, the information terminal 120 extracts a character string from the corrected character.

[0210] In step S1107, the information terminal 120 determines whether the extracted character string is described in accordance with the naming rule of the object item.

[0211] In step S1108, the information terminal 120, in a case where it is determined that the character string is not described in accordance with the naming rule (in a case where "No" in step S1108), proceeds to step S1110.

[0212] On the other hand, in step S1108, the information terminal 120, in a case where it is determined that the character string is described in accordance with the naming rule (in a case where "Yes" in step S1108), proceeds to step S1109.

[0213] In step S1109, the information terminal 120 displays the character string determined to be described in accordance with the naming rule (the character recognition result) together with a frame.

[0214] In step S1110, the information terminal 120 performs the first storage control processing. Note that the details of the first storage control processing have already been described with reference to FIG. 6, and thus the description thereof is omitted here. Figure 8

[0215] In step S1111, the information terminal 120 determines whether the shutter operation is performed. When it is determined in step S1111 that the shutter operation is not performed (in a case where "No" in step S1111), the process returns to step S1102.

[0216] On the other hand, when it is determined in step S1111 that the shutter operation is performed (in a case where "Yes" in step S1111), the process proceeds to step S1112.

[0217] In step S1112, the information terminal 120 performs the second storage control processing. Note that the details of the second storage control processing have already been described with reference to FIG. 7, and thus the description thereof is omitted here. Figure 9

[0218] In step S1113, the information terminal 120 superimposes and displays the determined character recognition result together with a frame on the determined frame image.

[0219] <Summary>

[0220] As is apparent from the above description, the information terminal 120 of the first embodiment ​​

[0221] • The process of acquiring each frame image included in the captured moving image data and the process of storing the character recognition result acquired by performing the character recognition process on the frame image to the first storage section are repeatedly performed.

[0222] • The character recognition result is determined in a case where the shutter operation by the maintenance worker is accepted.

[0223] Thus, in the capturing of the moving image data, the character recognition process is performed on each frame image, and at the time point when the shutter operation is performed, the character recognition result that is correctly acquired is determined. Therefore, according to the first embodiment, the maintenance worker does not have to newly perform the capturing for acquiring the device data.

[0224] Therefore, according to the first embodiment, it is possible to reduce the work load of the worker when acquiring the device data of the site device by capturing.

[0225] [Second Embodiment]

[0226] In the above-described first embodiment, the maintenance worker can perform the shutter operation regardless of whether the device data can be correctly acquired in the capturing of the moving image data, but it can also be configured so that the shutter operation cannot be performed until the device data is correctly acquired. Hereinafter, the second embodiment will be described focusing on the difference from the above-described first embodiment.

[0227] <Function Structure of Information Terminal>

[0228] Figure 12 is a second drawing showing an example of the function structure of the information terminal. The function structure of the information terminal 120 of the second embodiment is different from the function structure of the information terminal 120 of the first embodiment described above in that the function of the operation accepting section 1201 is different from the function of the operation accepting section 401. Figure 4

[0229] In the information terminal 120 of the second embodiment, the operation accepting section 1201 accepts the capturing start operation by the maintenance worker 130, notifies the acquisition section 402 of the capturing start instruction, and starts monitoring the first storage section 425.

[0230] Specifically, the operation accepting section 1201 monitors whether the character recognition result on the target item (model name, production number, production date) is stored in the first storage section 425. When it is determined that the character recognition result is not stored, the operation accepting section 1201 does not accept the shutter operation by the maintenance worker 130. On the other hand, when it is determined that the character recognition result is stored, the operation accepting section 1201 accepts the shutter operation by the maintenance worker 130 and notifies the second storage control section 409 that the shutter operation is performed. ​

[0231] <Summary>

[0232] As is apparent from the above description, the information terminal 120 according to the second embodiment is configured not to perform the shutter operation until the device data can be correctly acquired after the photographing start operation is performed.

[0233] Thus, according to the second embodiment, the maintenance worker can reliably eliminate the re-photographing.

[0234] [Other Embodiments]

[0235] In the above-described first and second embodiments, the case where the shutter operation is assigned as the operation for determining the character recognition result is described, but an operation other than the shutter operation can be assigned as the operation for determining the character recognition result.

[0236] Further, in the above-described first and second embodiments, the character recognition result is determined by ending the photographing of the moving image data by performing the shutter operation. However, the configuration can be such that the photographing end of the moving image data and the determination of the character recognition result are automatically performed with the condition that the character recognition result on the target item is stored in the first storage section 425.

[0237] Further, in the above-described first and second embodiments, the case where the character recognition result is stored in the first storage section 425 when the character recognition result is obtained on all the target items is described. However, the method of storing in the first storage section 425 is not limited to this. For example, as in the above-described first and second embodiments, the configuration can be such that, in the case where a plurality of target items are included and the character recognition result is obtained on any one of the target items, the obtained character recognition result is overwritten as the character recognition result of the corresponding target item. That is, the configuration can be such that the character recognition result is stored individually in units of target items.

[0238] Further, in the above-described first and second embodiments, the case where the device data is read from the device nameplate is described, but the target of reading the device data is not limited to the device nameplate. Or, the target read from the device nameplate is not limited to the device data. Further, the target photographed by the imaging device is not limited to the device nameplate, but can be a target other than the device nameplate as long as the target includes the specific character string.

[0239] Further, in the above-described first and second embodiments, the device data rule in the case where the field device is the outdoor unit of the air conditioning device is exemplified, but in the case where the field devices are different from each other, the device data rule corresponding to the different field devices is used. Also, in the above-described first and second embodiments, the tag in the case where the field device is the outdoor unit of the air conditioning device is exemplified, but in the case where the field devices are different from each other, the tag corresponding to the different field devices is used.

[0240] Further, in the above-described first and second embodiments, for the sake of convenience of explanation, the case where the character recognition result obtained from the frame image is displayed on the frame image is explained. However, since it takes a certain time to obtain the character recognition result and to overlap it, the frame image on which the character recognition result is obtained and the frame image on which the obtained character recognition result is overlapped can also be different.

[0241] Although the embodiments have been described above, it is understood that various changes in form and details can be made without departing from the purpose and scope of the claims.

[0242] This application is based on Japanese Patent Application No. 2022-140707 filed on September 5, 2022, the contents of which are incorporated herein by reference in its entirety.

[0243] Explanation of Reference Signs

[0244] 110: Field device

[0245] 111: Device tag

[0246] 120: Information terminal

[0247] 121: Captured image

[0248] 305: Imaging device

[0249] 401: Operation accepting section

[0250] 402: Obtaining section

[0251] 403: Recognition section

[0252] 404: Correction section

[0253] 405: Extraction section

[0254] 406: Determining section

[0255] 407: Frame generating section

[0256] 408: First storage control section

[0257] 409: Second storage control section

[0258] 410: display section

[0259] 411: manual correction section

[0260] 1201: operation accepting section

Claims

1. An information terminal having a control section, wherein the control section: repeatedly performs a process of acquiring a captured frame image; and a process of storing, as a character recognition result judged to be a character string described in accordance with a naming rule, a character recognition result obtained by performing character recognition on the frame image in a storage section, and in a case where a prescribed operation for ending the capturing of a frame image performed on the information terminal is accepted and a character recognition result based on a frame image at the time when the prescribed operation is accepted is not judged to be described in accordance with the naming rule and is not stored in the storage section, determines a character recognition result based on a character recognition result based on a frame image before the frame image at the time when the prescribed operation is accepted, which is a character string judged to be described in accordance with the naming rule and stored in the storage section.

2. The information terminal according to claim 1, wherein in a case where the captured frame image is displayed in real time, the character recognition result obtained by performing the character recognition is displayed in superposition.

3. The information terminal according to claim 2, wherein the control section: updates the displayed frame image at a first time interval, and updates the character recognition result displayed in superposition at a second time interval longer than the first time interval.

4. The information terminal according to claim 1, wherein the control section: in a case where the captured frame image contains a plurality of items to be stored as character recognition results, stores a character recognition result of each item obtained by performing the character recognition in the storage section in units of items, in a case where a prescribed operation for ending the capturing of a frame image performed on the information terminal is accepted and a character recognition result based on a frame image at the time when the prescribed operation is accepted is not judged to be described in accordance with the naming rule and is not stored in the storage section, determines a character recognition result in units of items based on a character recognition result based on a frame image before the frame image at the time when the prescribed operation is accepted, which is a character string judged to be described in accordance with the naming rule and stored in the storage section, and the naming rule is a rule that specifies a character type, a number of characters, and an arrangement order of a character string constituting the item.

5. The information terminal according to claim 1, wherein the control section: in a case where the displayed character recognition result is corrected in correspondence with the display of the determined character recognition result on the display section, stores the corrected character recognition result in the storage section.

6. The information terminal according to claim 1, wherein the control section: obtains a model name and a production number as character recognition results by performing character recognition on a frame image in which a nameplate of an apparatus is captured.

7. The information terminal according to claim 1, wherein the control section: stores, as a character recognition result judged to be a character string described in accordance with a naming rule, a character recognition result obtained by performing the character recognition in a first storage section, and ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The character recognition result determined by accepting the prescribed operation for ending the photographing of the frame image performed on the information terminal is stored in the second storage section.

8. The information terminal according to claim 7, wherein the control section: updates the character recognition result stored in the first storage section each time a character recognition result is obtained as a character string determined to have been described in accordance with the naming rule by performing the character recognition.

9. The information terminal according to claim 7, wherein the control section: stores the frame image in which the character recognition result stored in the first storage section is obtained in the first storage section in association with the character recognition result, stores the frame image in which the character recognition result stored in the second storage section is obtained in the second storage section in association with the character recognition result.

10. A character recognition method, wherein, The following processes are performed by the control section of the information terminal: repeatedly performing a process of acquiring a photographed frame image; and a process of storing a character recognition result obtained by performing character recognition on the frame image as a character string determined to have been described in accordance with a naming rule in a storage section; and in a case where the prescribed operation for ending the photographing of the frame image performed on the information terminal is accepted and a character recognition result based on the frame image at the time when the prescribed operation is accepted is determined not to have been described in accordance with the naming rule and is not stored in the storage section, a character recognition result is determined based on a character recognition result based on a frame image prior to the frame image at the time when the prescribed operation is accepted as a character string determined to have been described in accordance with the naming rule and stored in the storage section.

11. A character recognition program product for causing a control section of an information terminal to perform the following processes: repeatedly performing processing to acquire the captured frame image; a process of acquiring a photographed frame image; and a process of storing a character recognition result obtained by performing character recognition on the frame image as a character string determined to have been described in accordance with a naming rule in a storage section; and in a case where the prescribed operation for ending the photographing of the frame image performed on the information terminal is accepted and a character recognition result based on the frame image at the time when the prescribed operation is accepted is determined not to have been described in accordance with the naming rule and is not stored in the storage section, a character recognition result is determined based on a character recognition result based on a frame image prior to the frame image at the time when the prescribed operation is accepted as a character string determined to have been described in accordance with the naming rule and stored in the storage section.

Images