Information processing device, specification method, and specification program

Abstract

An information processing device (100) comprises: an acquisition unit (120) that acquires an image (10) including numbers; a region detection unit (130) that uses the image (10) to detect a region (11) in which numbers are displayed; a determination / detection unit (150) that detects, from the detected region (11), a right segment that is a segment on a right side among a plurality of segments constituting the numbers; and a specification unit (170) that specifies, as a character region, a region in which the right segment is included.

Description

Information processing device, identification method, and identification program

[0001] The present disclosure relates to an information processing device, an identification method, and an identification program.

[0002] Meters are installed in buildings, etc. For example, the meter is an electricity meter. Numbers are displayed on the meter. For example, a user reads the numbers using a smartphone. Here, a technology for recognizing characters such as numbers has been proposed (see Patent Document 1). In Patent Document 1, a search area is set for an image to be recognized. The information processing device of Patent Document 1 sets clipping areas at multiple locations within the search area. The information processing device extracts images corresponding to each of the set clipping areas. The information processing device compares each of the extracted images with dictionary data to detect candidate character information and position information of the clipping areas corresponding to the candidate characters. The information processing device outputs the candidate character information with the highest evaluation value from the detected candidate character information as a recognition result.

[0003] Japanese Patent Application Laid-Open No. 2021-93225

[0004] When a user reads numbers, the character area where the numbers exist is identified before character recognition is performed. For example, when the character area is identified using the above-mentioned technology, many segmentation areas are set and the character area is identified. However, this method takes a long time to identify the character area.

[0005] An object of the present disclosure is to identify a character region in a short time.

[0006] According to one aspect of the present disclosure, there is provided an information processing device including: an acquisition unit that acquires an image including numbers; a region detection unit that uses the image to detect an area in which the numbers are displayed; a determination and detection unit that detects, from the detected area, a right-side segment that is a right-side segment among multiple segments that make up the numbers; and an identification unit that identifies, as a character area, an area including the right-side segment.

[0007] According to the present disclosure, a character region can be identified in a short amount of time.

[0008] 1 is a diagram showing a specific example of a user's work in embodiment 1. FIG. 2 is a diagram showing hardware possessed by the information processing device of embodiment 1. FIG. 3 is a block diagram showing the functions of the information processing device of embodiment 1. FIG. 4 is a diagram showing a specific example (part 1) of the search process of embodiment 1. FIG. 5 is a diagram showing a specific example (part 2) of the search process of embodiment 1. FIG. 6 is a flowchart showing an example of the process executed by the information processing device of embodiment 1. FIG. 7 is a diagram showing an example of a case where step S15 of embodiment 1 is not executed. FIG. 8 is a block diagram showing the functions of the information processing device of embodiment 2. FIG. 9 is a flowchart showing an example (part 1) of the process executed by the information processing device of embodiment 2. FIG. 10 is a flowchart showing an example (part 2) of the process executed by the information processing device of embodiment 2. (A) to (C) are diagrams showing a specific example of the search process of embodiment 2.

[0009] Hereinafter, embodiments will be described with reference to the drawings. The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1. Fig. 1 is a diagram showing a specific example of a user's work in embodiment 1. Fig. 1 shows an information processing device 100. The information processing device 100 is a device that executes an identification method. The information processing device 100 is a smartphone, a tablet device, or the like. The meter 200 has a seven-segment display. The user uses the information processing device 100 to capture an image of the meter 200. As a result, the information processing device 100 can obtain an image including numbers.

[0011] Next, the hardware of the information processing device 100 will be described. Fig. 2 is a diagram showing the hardware of the information processing device of embodiment 1. The information processing device 100 is a computer. The information processing device 100 has a processor 101, a volatile storage device 102, a non-volatile storage device 103, and a camera 104.

[0012] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.

[0013] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a RAM (Random Access Memory). The nonvolatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the nonvolatile storage device 103 is an SSD (Solid State Drive). The camera 104 is a device that captures images.

[0014] Next, a description will be given of the functions of the information processing device 100. Fig. 3 is a block diagram showing the functions of the information processing device of embodiment 1. The information processing device 100 has a storage unit 110, an acquisition unit 120, a region detection unit 130, a conversion unit 140, a determination and detection unit 150, a change unit 160, and an identification unit 170.

[0015] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103. Some or all of the acquisition unit 120, area detection unit 130, conversion unit 140, determination and detection unit 150, change unit 160, and identification unit 170 may be realized by a processing circuit. Also, some or all of the acquisition unit 120, area detection unit 130, conversion unit 140, determination and detection unit 150, change unit 160, and identification unit 170 may be realized as program modules executed by the processor 101. For example, the program executed by the processor 101 is also referred to as a specific program. For example, the specific program is recorded on a recording medium.

[0016] The storage unit 110 stores various information. The acquisition unit 120 acquires an image including numbers represented by segments. For example, the acquisition unit 120 acquires the image generated by the camera 104 from the camera 104. The acquisition unit 120 may also acquire the image from another device.

[0017] The area detection unit 130 uses the image to detect an area where numbers are displayed. For example, the area detection unit 130 detects the area using feature point matching. Furthermore, when using feature point matching, the area detection unit 130 may perform projective transformation on the image to match it with a template image. When projective transformation is performed, the area is an area where numbers are displayed in the image transformed by projective transformation. The conversion unit 140 binarizes the area where numbers are displayed. The binarization allows the information processing device 100 to easily identify the position of the segment.

[0018] The acquisition unit 120 acquires character recognition candidate area information indicating the character recognition candidate area. For example, the acquisition unit 120 acquires the character recognition candidate area information from the storage unit 110. Also, for example, the acquisition unit 120 acquires the character recognition candidate area information from an external device. Note that the external device is a device that exists outside the information processing device 100. For example, the external device is a cloud server. An illustration of the external device is omitted. The character recognition candidate area is an area of ​​a predetermined size. For example, the character recognition candidate area is represented by a plurality of coordinates in an area where numbers are displayed.

[0019] Here, among the multiple segments that make up a number, some segments on the right side always exist. Therefore, the information processing device 100 searches for the right-side segments. The right-side segments are called right-side segments. Also, the right-side segments are one or more vertical segments on the right side. The search process will be explained using a specific example.

[0020] 4 shows a specific example (part 1) of the search process of embodiment 1. The upper diagram in FIG. 4 shows image 10. Image 10 includes an area 11 in which numbers are displayed. The lower diagram in FIG. 4 shows character recognition candidate areas 20a to 20c.

[0021] 5 is a diagram showing a specific example (part 2) of the search process of the first embodiment. When multiple numbers are present in an image, the number with a right-side segment to be searched for is one of the multiple numbers. In the following description, the number with a right-side segment is the rightmost number among the multiple numbers.

[0022] The determination and detection unit 150 determines whether a right-side segment is present to the right of the character recognition candidate region 20a in the region 11. Specifically, if a vertical edge is present in the determination region 30a, which is the region to the right of the character recognition candidate region 20a, the determination and detection unit 150 determines that a right-side segment is present in the determination region 30a. Note that, for example, the determination region 30a is an area outside the character recognition candidate region 20a and is one-third of the character recognition candidate region 20a. The determination and detection unit 150 may also perform template matching on the determination region 30a to determine whether a right-side segment is present in the determination region 30a.

[0023] When no right-side segment exists to the right of the character recognition candidate region 20a, the determination and detection unit 150 determines whether or not a right-side segment exists in the right region within the character recognition candidate region 20a. Specifically, when a vertical edge exists in a determination region 30b, which is the right region within the character recognition candidate region 20a, the determination and detection unit 150 determines that a right-side segment exists in the right region. Note that, for example, the determination region 30b is one-third of the character recognition candidate region 20a. The determination and detection unit 150 may also perform template matching on the determination region 30b to determine whether or not a right-side segment exists in the right region.

[0024] If a right-side segment does not exist in the right region of the character recognition candidate region 20a, the determination and detection unit 150 detects a right-side segment from the region to the left of the right region. Specifically, if a vertical edge exists in a determination region 30c, which is the region to the left of the right region, the determination and detection unit 150 determines that a right-side segment exists in the determination region 30c. Note that, for example, the determination region 30c is one-third of the character recognition candidate region 20a. The determination and detection unit 150 may also perform template matching on the determination region 30c to determine whether or not a right-side segment exists in the determination region 30c. Furthermore, if a right-side segment does not exist in the determination region 30c, the determination and detection unit 150 detects a right-side segment from the region to the left of the determination region 30c.

[0025] In this way, the determination detection unit 150 searches the area 11. Then, the determination detection unit 150 detects the right-side segment from the area 11.

[0026] If a right-side segment is present to the right of character recognition candidate region 20a, modification unit 160 moves character recognition candidate region 20a so that the right-side segment is included in character recognition candidate region 20a. If a right-side segment is present in the region to the left of the right region, modification unit 160 moves character recognition candidate region 20a so that the right-side segment is included in the right region. Then, modification unit 160 moves character recognition candidate regions 20b and 20c corresponding to numbers other than those containing a right-side segment based on the amount of movement of character recognition candidate region 20a.

[0027] The identification unit 170 identifies an area that includes the right-side segment as a character area. Specifically, if the right-side segment is included in the right area, the identification unit 170 identifies the character recognition candidate areas 20a to 20c corresponding to multiple numbers as character areas. Furthermore, if the character recognition candidate areas 20a to 20c have moved, the identification unit 170 identifies the moved character recognition candidate areas 20a to 20c as character areas.

[0028] Next, the processing executed by the information processing device 100 will be described using a flowchart. FIG. 6 is a flowchart showing an example of the processing executed by the information processing device of embodiment 1. (Step S11) The acquisition unit 120 acquires an image including numbers represented by segments. (Step S12) The area detection unit 130 uses the image to detect an area in which the numbers are displayed. (Step S13) The conversion unit 140 binarizes the area in which the numbers are displayed. (Step S14) The acquisition unit 120 acquires character recognition candidate area information.

[0029] (Step S15) The determination and detection unit 150 determines whether or not a right-side segment exists to the right of the character recognition candidate area. If a right-side segment exists, the process proceeds to step S18. If a right-side segment does not exist, the process proceeds to step S16. (Step S16) The determination and detection unit 150 determines whether or not a right-side segment exists in the right area within the character recognition candidate area. If a right-side segment does not exist, the process proceeds to step S17. If a right-side segment exists, the process proceeds to step S19.

[0030] (Step S17) The determination and detection unit 150 determines whether or not a right-side segment is present to the left of the right region. If a right-side segment is present, the process proceeds to step S18. If a right-side segment is not present, the information processing device 100 may display a message on the display of the information processing device 100 prompting the user to re-capture the image. (Step S18) The change unit 160 changes the character recognition candidate region based on the position of the right-side segment. The change unit 160 moves other character recognition candidate regions based on the amount of movement of the character recognition candidate region. (Step S19) The identification unit 170 identifies the region containing the right-side segment as a character region.

[0031] After the character region is identified, the information processing device 100 performs character recognition on the character region.

[0032] According to the first embodiment, the information processing device 100 performs a maximum of three determinations to search for a right-side segment. In other words, the information processing device 100 can identify a character region with a maximum of three determinations. That is, the information processing device 100 identifies a character region with a small number of determinations. Therefore, the information processing device 100 can identify a character region in a short amount of time.

[0033] The above describes a case where binarization is performed. The information processing device 100 does not have to perform binarization. The above describes a case where three numbers are included in an image. However, the first embodiment is also effective when a single character is included in an image.

[0034] Furthermore, the information processing device 100 does not have to execute step S15. That is, the information processing device 100 may execute step S16 after step S14. However, it is preferable that step S15 be executed. The reason for this will be explained using the drawings.

[0035] FIG. 7 is a diagram showing an example in which step S15 in embodiment 1 is not executed. The upper diagram in FIG. 7 shows character recognition candidate areas 40a to 40c. If step S15 is not executed, the information processing device 100 executes step S16 after step S14. The information processing device 100 determines whether or not a right-side segment is present in the determination area 50, which is the right area within the character recognition candidate area 40a. If a determination is made in this manner, the information processing device 100 makes an erroneous determination. Therefore, the information processing device 100 executes step S15. By executing step S15, the information processing device 100 can prevent an erroneous determination.

[0036] Second Embodiment Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be mainly described. Furthermore, in the second embodiment, descriptions of the commonalities between the first embodiment and the second embodiment will be omitted.

[0037] In the first embodiment, a case where a character area is specified has been described. More specifically, in the method of the first embodiment, the position of the character area in the horizontal direction is specified. Here, the accuracy of the position of the character area in the vertical direction is a concern. Therefore, a method of repeatedly searching for the position of the character area in the vertical direction can be considered. However, performing searches multiple times increases the load on the information processing device 100. Therefore, in the second embodiment, a case where a limit is placed on the number of searches will be described.

[0038] 8 is a block diagram showing functions of the information processing device of embodiment 2. The information processing device 100 further includes a detection unit 180, a determination unit 181, and a search unit 182. Some or all of the detection unit 180, the determination unit 181, and the search unit 182 may be realized by a processing circuit. Also, some or all of the detection unit 180, the determination unit 181, and the search unit 182 may be realized as program modules executed by the processor 101. The functions of the detection unit 180, the determination unit 181, and the search unit 182 will be described later.

[0039] Next, the processing executed by the information processing device 100 will be described using flowcharts. Fig. 9 is a flowchart showing an example (part 1) of the processing executed by the information processing device of embodiment 2. Fig. 10 is a flowchart showing an example (part 2) of the processing executed by the information processing device of embodiment 2.

[0040] 9 and 10 differ from the process in Fig. 6 in that steps S12a, 12b, and 19a are executed. Therefore, steps S12a, 12b, and 19a will be described in Fig. 9 and 10. Description of the processes other than steps S12a, 12b, and 19a will be omitted.

[0041] (Step S12a) The detection unit 180 detects edges based on the area where the numbers are displayed. For example, the detection unit 180 detects edges using a Laplacian filter. The detection unit 180 may also detect brightness based on the area where the numbers are displayed.

[0042] (Step S12b) The determination unit 181 determines the number of searches based on the amount of edge detection. Specifically, the greater the amount of edge detection, the smaller the value set by the determination unit 181 for the number of searches. The smaller the amount of edge detection, the larger the value set by the determination unit 181 for the number of searches. For example, when the amount of edge detection is large, the determination unit 181 sets the number of searches to "3." Furthermore, the determination unit 181 may determine the number of searches using information indicating the correspondence between the amount of edge detection and the number of searches.

[0043] Furthermore, the determination unit 181 may determine the number of searches based on the brightness. For example, the greater the brightness, the smaller the value set by the determination unit 181 for the number of searches. For example, the lower the brightness, the larger the value set by the determination unit 181 for the number of searches. Furthermore, the determination unit 181 may determine the number of searches using information indicating the correspondence between brightness and the number of searches.

[0044] (Step S19a) The search unit 182 searches for the position of the character area in the vertical direction based on the number of searches. A specific example of the search process will be shown below.

[0045] 11A to 11C are diagrams showing a specific example of the search process of the second embodiment. The number of searches is assumed to be three. FIG. 11A shows an example of the first search. The search unit 182 determines whether all segments constituting the number are included in the character regions 60a to 60c identified in step S19. Specifically, the search unit 182 makes this determination using template matching. Alternatively, the search unit 182 may make this determination using a trained model.

[0046] FIG. 11B shows an example of the second search. The search unit 182 shifts the character regions 60a to 60c upward. The shift distance is determined in advance. The search unit 182 determines whether all segments that make up the number are included in the shifted character regions 60a to 60c. Specifically, the search unit 182 makes this determination using template matching. Alternatively, the search unit 182 may make this determination using a trained model.

[0047] 11C shows an example of the third search. The search unit 182 shifts the character regions 60a to 60c downward. The search unit 182 determines whether all segments that make up the number are included in the shifted character regions 60a to 60c. Specifically, the search unit 182 makes this determination using template matching. Alternatively, the search unit 182 may make this determination using a trained model.

[0048] The search unit 182 determines the position of the vertical character region where the similarity obtained by template matching is highest. Furthermore, when a trained model is used, the search unit 182 determines the position of the vertical character region based on the results output by the trained model. In the case of Figure 11, the search unit 182 determines the position of Figure 11(A) as the position of the vertical character region.

[0049] According to the second embodiment, the information processing device 100 imposes a limit on the number of searches. Therefore, the information processing device 100 does not perform searches multiple times. Therefore, the information processing device 100 can reduce the load on the information processing device 100.

[0050] The features of the above-described embodiments can be combined with each other as appropriate.

[0051] 10 image, 11 area, 20a to 20c character recognition candidate area, 30a to 30c judgment area, 40a to 40c character recognition candidate area, 50 judgment area, 60a to 60c character area, 100 information processing device, 101 processor, 102 volatile storage device, 103 non-volatile storage device, 104 camera, 110 storage unit, 120 acquisition unit, 130 area detection unit, 140 conversion unit, 150 judgment detection unit, 160 change unit, 170 identification unit, 180 detection unit, 181 decision unit, 182 search unit, 200 meter.

Claims

1. An information processing device having: an acquisition unit that acquires an image including numbers; an area detection unit that uses the image to detect an area in which the numbers are displayed; a judgment and detection unit that detects, from the detected area, a right-side segment that is the right-side segment of multiple segments that make up the number; and an identification unit that identifies an area including the right-side segment as a character area.

2. The information processing device according to claim 1, wherein the acquisition unit acquires character recognition candidate area information indicating a character recognition candidate area, and the determination detection unit determines whether or not the right-side segment is present in a right area within the character recognition candidate area within the detected area.

3. The information processing device according to claim 2, wherein, when the right-side segment is not detected by the determination, the determination detection unit detects the right-side segment from within an area to the left of the right area.

4. The information processing device of claim 2, wherein the judgment and detection unit judges whether the right-side segment is present to the right of the character recognition candidate area within the detected area, and if the right-side segment is not detected by the judgment, judges whether the right-side segment is present in the right area within the character recognition candidate area.

5. An information processing device according to any one of claims 2 to 4, further comprising a modification unit, wherein when a plurality of numerals are present in the image, the numeral is one of the plurality of numerals, the modification unit moves the character recognition candidate region so that the right side segment is included in the character recognition candidate region corresponding to the numeral when the right side segment is present to the right of the character recognition candidate region corresponding to the numeral, moves the character recognition candidate region so that the right side segment is included in the right region when the right side segment is present in the region to the left of the right region, and moves the character recognition candidate region corresponding to numerals other than the numeral based on the amount of movement of the character recognition candidate region, and the identification unit identifies the plurality of character recognition candidate regions corresponding to the plurality of numerals as the character region when the right side segment is included in the right region, and identifies the moved plurality of character recognition candidate regions as the character region when the plurality of character recognition candidate regions have been moved.

6. An information processing device as described in any one of claims 1 to 5, further comprising: a detection unit that detects edges or brightness based on the detected area; a determination unit that determines the number of searches based on the amount of edge detection or the brightness; and a search unit that searches for the position of the character area in the vertical direction based on the number of searches.

7. The information processing device according to claim 1, further comprising a conversion unit that binarizes the detected region.

8. An identification method comprising: an information processing device acquiring an image including numbers; detecting an area in which the numbers are displayed using the image; detecting a right-side segment, which is the right-side segment among a plurality of segments constituting the number, from the detected area; and identifying an area including the right-side segment as a character area.

9. A specific program that causes an information processing device to execute the following process: acquire an image including numbers; use the image to detect an area in which the numbers are displayed; detect a right-side segment, which is the right-side segment among multiple segments that make up the number, from the detected area; and identify the area including the right-side segment as a character area.

Images