Character recognition device and character recognition program

Abstract

To provide a character recognition device and a character recognition program that hardly allow poor quality image data to be subjected to character recognition processing.SOLUTION: In a character recognition device, an evaluation index calculation unit 143 acquires a plurality of noise elimination processed data D3 arranged in time series generated by repeatedly imaging a display screen on which characters are displayed to calculate an evaluation index showing excellent image quality for each of the acquired noise elimination processed data D3. A selection unit 144 selects high-quality image data D5 that is the noise elimination processed data D3 from which an evaluation index is calculated indicating that image quality is highest in an image data group D4 for each image data group D4 composed of the noise elimination processed data D3 whose character recognition period is longer than a time interval between the noise elimination processed data D3 arranged in time series. A character recognition unit 146 performs character recognition processing for recognizing characters from the high-quality image data D5 subjected to ruled line removal processing.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a character recognition device and a character recognition program. [Background technology]

[0002] A character recognition device that reads characters representing measurement results from the display screen of a measuring instrument is known, as disclosed in Patent Document 1. This character recognition device includes an imaging device that captures an image of the display screen of the measuring instrument, and a character recognition unit that performs character recognition processing to recognize characters representing the measurement results using image data generated by the imaging device.

[0003] In this specification, the concept of "characters" also includes "numbers." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-197851 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, there are cases where it is desired to increase the reliability of character recognition by performing character recognition processing multiple times and taking a majority vote of the results. There are also cases where it is desired to output the results of character recognition processing to an external device one after another in chronological order. In such cases, character recognition processing is performed on each of the multiple image data obtained by capturing multiple images of the display screen of a measuring instrument.

[0006] However, among these multiple image data, there may be some that are of poor quality due to camera shake, out-of-focus images, noise, etc. Even if character recognition processing is performed on such image data of poor quality, it is difficult to say that the results of the character recognition processing contribute to improving the reliability. Furthermore, even if the results of the character recognition processing are output externally, it is difficult to say that they will be effectively used externally.

[0007] In this way, when all image data obtained from multiple captures is subjected to character recognition processing, it is possible that character recognition processing will be performed unnecessarily on image data with poor image quality, which can result in a longer processing time until the majority vote result is obtained and an increased load on the processor.

[0008] Furthermore, when the imaging repetition period is sufficiently shorter than the character recognition processing repetition period, it may be desirable to suppress the use of image data of low image quality in the character recognition processing, and selectively use only image data of relatively high image quality in the character recognition processing.

[0009] An object of the present invention is to provide a character recognition device and a character recognition program that are less likely to subject poor quality image data to character recognition processing. [Means for solving the problem]

[0010] The character recognition device according to the present invention comprises: an evaluation index calculation unit that acquires a plurality of pieces of image data arranged in time series, which are generated by repeatedly capturing images of a display screen displaying characters, and calculates an evaluation index representing the quality of the image data for each of the acquired image data; a selection unit that selects, for each image data group consisting of image data corresponding to a character recognition cycle that is a predetermined time interval longer than the time interval between the image data arranged in time series, high-quality image data, which is the image data for which the evaluation index that represents the best image quality in the image data group is calculated by the evaluation index calculation unit; a character recognition unit that performs character recognition processing to recognize characters from each of the high-quality image data selected by the selection unit for each of the image data groups; Equipped with 、 the selection unit selects the high-quality image data for each character recognition cycle; the character recognition unit recognizes the characters from the high-quality image data each time the selection unit selects the high-quality image data; The character recognition cycle is equal to or less than the period required for the character recognition unit to recognize the character from one of the high-quality image data.

[0011] a noise removal unit that acquires a plurality of original image data arranged in time series, which are the source of the plurality of image data arranged in time series, and performs a noise removal process on each of the acquired original image data to remove noise components representing external light reflected on the display screen; Furthermore, The image data that is the subject of calculation of the evaluation index by the evaluation index calculation unit may be noise-removed data obtained by the noise removal unit applying the noise removal process to the original image data.

[0012] The noise removal process A process of creating blurred data by performing blurring on the original image data; A process of subtracting the blurred data from the original image data; may include:

[0013] a pre-processing unit that acquires a plurality of pieces of raw image data arranged in time series from an imaging device that repeatedly captures images of the display screen, and performs a cropping process on each of the acquired raw image data to remove peripheral areas other than the display screen, and a homography process on the raw image data that has been subjected to the cropping process to approximate a form in which the display screen is captured from directly in front of the image; Furthermore, The original image data that is the subject of the noise removal processing by the noise removal unit may be preprocessed data obtained by the preprocessing unit performing the cutout processing and the homography processing on the raw image data.

[0014] A ruled line is displayed on the display screen together with the characters, a ruled line removal unit that performs a ruled line removal process on each of the high-quality image data to remove components representing the ruled lines from the high-quality image data; Furthermore, The character recognition unit may perform the character recognition process on the high-quality image data that has been subjected to the ruled line removal process by the ruled line removal unit.

[0015] an output unit that uses the results of the character recognition process for each of the high-quality image data to identify a final reading result of the characters under the condition that the result of the character recognition process with the highest appearance frequency is adopted, and outputs read-result character data representing the identified reading result; may further comprise:

[0016] a display device that displays an imaging area including the display screen so that the content of the imaging can be confirmed when imaging the display screen; an overlay display control unit that causes the read result character data output by the output unit to be displayed on the display device by overlaying it on the imaging area; may further comprise:

[0017] an evidential image data specification unit that specifies, among all the image data arranged in chronological order, the image data for which the evaluation index indicating the best image quality is calculated by the evaluation index calculation unit as evidential image data, and controls the storage of the specified evidential image data; may further comprise:

[0018] The character recognition program according to the present invention is computer of , A plurality of image data items arranged in time series are acquired by repeatedly capturing images of a display screen displaying characters, and an evaluation index representing the quality of the image data is calculated for each of the acquired image data items. Department , For each image data group consisting of the image data for a character recognition period that is a predetermined time interval longer than the time interval between the image data arranged in time series, the evaluation index that represents the best image quality in the image data group is calculated. Department The image data of high image quality calculated by the above method is selected. Department , The selection Department character recognition processing for recognizing the characters from each of the high-quality image data selected by Department , It functions as the selection unit selects the high-quality image data for each character recognition cycle; the character recognition unit recognizes the characters from the high-quality image data each time the selection unit selects the high-quality image data; The character recognition cycle is equal to or less than the period required for the character recognition unit to recognize the character from one of the high-quality image data. [Effects of the Invention]

[0019] According to the above configuration, the high-quality image data, for which an evaluation index representing the highest image quality is calculated, is subjected to character recognition processing. Therefore, image data with low image quality is less likely to be subjected to character recognition processing. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing the configuration of a fisheries management support system according to an embodiment. [Figure 2] 1 is a conceptual diagram showing the configuration of a character recognition device according to an embodiment. [Figure 3] FIG. 2 is a conceptual diagram showing the functions of the character recognition device according to the embodiment. [Figure 4] (A): A conceptual diagram showing raw image data according to an embodiment. (B): A conceptual diagram showing a boundary display member extracted from the raw image data. (C): A conceptual diagram showing the results of performing a cutout process on the raw image data. (D): A conceptual diagram showing the results of performing a homography process on the raw image data. [Figure 5] 1A is a conceptual diagram showing preprocessed data according to an embodiment, and FIG. 1B is a conceptual diagram showing preprocessed data after noise removal processing according to an embodiment. [Figure 6] (A): A conceptual diagram showing high-quality image data according to an embodiment. (B): A conceptual diagram showing components representing ruled lines in high-quality image data according to an embodiment. (C): A conceptual diagram showing data after ruled line removal processing according to an embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing a display screen of a display device on which read result character data is overlaid according to the embodiment. [Figure 8] 10 is a flowchart of a character reading process according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, a fisheries management support system according to an embodiment will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals.

[0022] As shown in Figure 1, the fisheries management support system 300 of this embodiment includes a character recognition device 100 operated by a user engaged in fishing on a fishing vessel FS, and a management server 200 capable of communicating with the character recognition device 100 via a communication line NE.

[0023] The ship FS is equipped with a tidal current meter MA that measures the flow velocity of tidal currents. The tidal current meter MA has a display screen DS that displays the measurement results. The tidal current meter MA is an example of a measuring instrument that measures various physical quantities.

[0024] The character recognition device 100 is configured as a portable computer, specifically a smartphone. The character recognition device 100 is operated by a user on board a ship to read characters such as numbers and symbols that represent measurement results from the display screen DS of the tidal current meter MA.

[0025] Then, the character recognition device 100 transmits read result character data D8 representing the characters read from the display screen DS to the management server 200 at the timing when communication with the management server 200 is established. The management server 200 uses the read result character data D8 acquired from the character recognition device 100 to create reports, logs, etc. that indicate the details of the operations performed on the vessel FS, the amount of catch, etc.

[0026] The most notable feature of the fisheries management support system 300 according to this embodiment is the configuration of the character recognition device 100. Therefore, the character recognition device 100 will be specifically described below.

[0027] As shown in Fig. 2, character recognition device 100 includes imaging device 110 capable of capturing moving images, and display device 120 that displays the content captured by imaging device 110. Imaging device 110 is used to capture an image of the display screen DS of tidal current meter MA shown in Fig. 1. When imaging device 110 captures an image, display device 120 displays and outputs the captured content in real time so that the user can confirm the content.

[0028] The character recognition device 100 also includes a storage device 130 in which a character recognition program 131 is stored, and a processor 140 that executes the character recognition program 131 .

[0029] The character recognition program 131 defines the procedure for a character reading process that uses the image pickup results from the imaging device 110 to read characters representing measurement results from the display screen DS of the tidal current meter MA shown in FIG.

[0030] The storage device 130 also stores template data 132 used for character recognition in the character reading process. The storage device 130 also stores evidence image data 133 representing the imaging results that were the subject of character recognition in the character reading process.

[0031] Character recognition device 100 also includes communication device 150 that communicates with management server 200 shown in Fig. 1. Communication device 150 transmits the results of characters read by processor 140 in accordance with the specifications of character recognition program 131 to management server 200 as read result character data D8 shown in Fig. 1.

[0032] The functions of the character recognition device 100 will be specifically described below.

[0033] 3, the imaging device 110 generates and outputs raw image data D1. The raw image data D1 represents the result of capturing an image of the display screen DS of the tidal current meter MA shown in FIG.

[0034] As described above, the imaging device 110 captures the display screen DS of the tidal current meter MA shown in FIG. 1 as a moving image. Specifically, the imaging device 110 repeatedly captures still images. In other words, the raw image data D1 represents a frame as a single still image that constitutes the moving image. While FIG. 3 shows a representative example of one frame of raw image data D1, the imaging device 110 outputs the raw image data D1 one after another in chronological order.

[0035] Next, a description will be given of the function of the processor 140. The processor 140 performs the function of a pre-processing unit 141 that acquires a plurality of chronologically arranged raw image data D1 from the imaging device 110 by executing the character recognition program 131 shown in FIG.

[0036] The pre-processing unit 141 performs a cropping process on each of the raw image data D1 acquired from the imaging device 110 to remove the peripheral area other than the display screen DS shown in Fig. 1, and a homography process to make the cropped raw image data D1 closer to the form of the display screen DS shown in Fig. 1 captured from directly in front. Each process will be described in detail below.

[0037] 4A shows an example of raw image data D1 acquired from the imaging device 110. Because it is difficult to capture only the area of ​​the display screen DS on a rocking ship, the raw image data D1 also includes a peripheral area PD surrounding the display screen DS. However, this peripheral area PD is unnecessary because it does not contribute to reading the characters. The process of removing this peripheral area PD is called the cropping process.

[0038] Furthermore, since it is difficult to capture the display screen DS from directly in front on a rocking ship, the raw image data D1 represents the result of capturing the image from an angle relative to the normal to the display screen DS. As a result, the display screen DS is distorted into a parallelogram or trapezoid rather than a rectangle. The process of correcting this distortion is called homography processing.

[0039] First, the cutout process will be described. To realize the cutout process, the area of ​​the display screen DS needs to be recognized by the preprocessing unit 141. To facilitate this recognition, a boundary display member CT indicating the edge, i.e., the boundary, of the display screen DS is attached to the tidal current meter MA in advance.

[0040] The boundary display member CT has a color that does not appear or has a relatively low probability of appearing within the display screen DS and the peripheral area PD. In this embodiment, the boundary display member CT is made of pink masking tape. If the area of ​​the boundary display member CT is recognized by the pre-processing unit 141, the area of ​​the display screen DS is also automatically recognized.

[0041] Fig. 4(B) shows the result of the boundary display member CT being recognized by the preprocessing unit 141. In Fig. 4(B), for the sake of convenience, in order to clearly show that the boundary display member CT is recognizable, the boundary display member CT is shown in white, and the area other than the boundary display member CT is shown in black.

[0042] However, it is not essential to create the image data shown in Fig. 4(B). As described above, the boundary display member CT has a color that does not appear or has a relatively low probability of appearing within the display screen DS and the peripheral area PD, so the preprocessing unit 141 can recognize the area of ​​the boundary display member CT by its color in the raw image data D1.

[0043] If the pre-processing unit 141 can recognize the area of ​​the boundary display member CT that surrounds the display screen DS in a frame-like shape in the raw image data D1, it can achieve the cut-out process by removing the area outside the area of ​​the boundary display member CT.

[0044] Fig. 4(C) shows raw image data D1 that has been subjected to the cutout process by the pre-processing unit 141. The cutout process has removed the peripheral area PD shown in Fig. 4(A).

[0045] However, the area of ​​the boundary display member CT still remains in the raw image data D1 shown in Figure 4(C). Also, as described above, since the raw image data D1 is the result of capturing an image of the display screen DS from an oblique angle, the display screen DS in the raw image data D1 is distorted into a parallelogram or trapezoid rather than a rectangle. Therefore, the pre-processing unit 141 further performs homography processing to remove the area of ​​the boundary display member CT and correct the distortion.

[0046] Specifically, first, the preprocessing unit 141 recognizes four corners C1, C2, C3, and C4 formed by the boundary between the boundary display member CT and the rectangular display screen DS.

[0047] Next, the preprocessing unit 141 leaves only the area of ​​a parallelogram or trapezoid formed by connecting the four corners C1 to C4 with straight lines, that is, the area of ​​the display screen DS.

[0048] Next, the preprocessing unit 141 performs homography processing to make the remaining parallelogram or trapezoidal region of the display screen DS closer to the shape of the display screen DS as seen in FIG. 1 when it is imaged from directly in front.

[0049] Fig. 4(D) shows raw image data D1 that has been subjected to homography processing. The parallelogram or trapezoidal area of ​​the display screen DS shown in Fig. 4(C) has been made closer to a rectangle in Fig. 4(D) by homography processing.

[0050] The raw image data D1 that has been subjected to the pre-processing described above, that is, the clipping process and the homography process, will be referred to as pre-processed data D2 below.

[0051] 3, the description will continue. The preprocessing unit 141 sequentially performs the preprocessing described above on each of the raw image data D1 acquired successively in time series from the imaging device 110, and outputs preprocessed data D2 successively in time series.

[0052] The processor 140 also functions as a noise removal unit 142 that removes noise from the preprocessed data D2 by executing the character recognition program 131 shown in FIG.

[0053] Specifically, the noise removal unit 142 successively acquires a plurality of pieces of preprocessed data D2 arranged in time series from the preprocessing unit 141, and performs noise removal processing on each of the acquired preprocessed data D2 to remove noise components representing external light reflected on the display screen DS shown in Fig. 1. The noise removal processing will be specifically described below.

[0054] 5A shows an example of preprocessed data D2 acquired from the preprocessing unit 141. In the preprocessed data D2, external light is reflected on the display screen DS.

[0055] 2, the "external light" refers to light that is incident on the display screen DS from the outside and then reflected by the display screen DS, thereby entering the imaging device 110. Due to the external light, blurred images representing, for example, the character recognition device 100, the user who is taking the picture, the scenery behind the user, etc. are reflected in the preprocessed data D2.

[0056] External light reflected on the display screen DS interferes with reading characters from the display screen DS. Therefore, the noise removal unit 142 performs noise removal processing to remove noise components representing external light reflected on the display screen DS.

[0057] Specifically, the noise removal unit 142 first performs blurring on the preprocessed data D2 to generate blurred data (hereinafter referred to as blurred data). Gaussian blur is preferably used as the blurring process.

[0058] The noise components representing external light are originally blurred, and therefore are less affected by blurring compared to the components representing the characters displayed on the display screen DS. For this reason, the noise components are left almost unchanged by the blurring process.

[0059] Next, the noise removal unit 142 subtracts the blurred data from the original preprocessed data D2. Here, "subtraction" means subtraction of corresponding pixel values. This subtraction removes noise components from the original preprocessed data D2. On the other hand, the subtraction of areas other than those containing noise components has almost no effect on the degree of clarity. In other words, the subtraction allows mainly noise components to be selectively removed.

[0060] 5B shows preprocessed data D2 that has been subjected to the above-described noise removal process. Hereinafter, the preprocessed data D2 that has been subjected to the noise removal process in this manner will be referred to as noise-removed data D3.

[0061] 3, the description will continue. The noise removal unit 142 sequentially performs the above-described noise removal process on each piece of preprocessed data D2 acquired successively in time series from the preprocessing unit 141, and outputs noise-removed data D3 successively in time series.

[0062] The processor 140 also functions as an evaluation index calculation unit 143 that calculates an evaluation index for each piece of noise-removed data D3 by executing the character recognition program 131 shown in FIG.

[0063] The evaluation index represents the quality of the image quality of the noise-removed data D3. Specifically, the evaluation index represents the degree of clarity of the image quality of the noise-removed data D3. To calculate the evaluation index, a Laplacian filter is used, which detects edges, which are areas where pixel values ​​change relatively significantly.

[0064] In this embodiment, the evaluation index calculation unit 143 calculates the evaluation index under the condition that the more edges with large spatial second derivative values ​​are scattered in the image with high contrast, the higher the degree of clarity of the image quality.

[0065] Specifically, the evaluation index calculation unit 143 first applies a Laplacian filter to the noise-removed data D3 to create Laplacian image data. Next, the evaluation index calculation unit 143 calculates the variance α of pixel values ​​in the Laplacian image data and the pixel value β with the highest brightness in the Laplacian image.

[0066] In this case, the evaluation index is defined as a value proportional to the product of α and β. As a specific example, the evaluation index is defined as α / 2×β.

[0067] The evaluation index calculation unit 143 calculates the evaluation index described above for each piece of noise-removed data D3 acquired in time series from the noise removal unit 142. The evaluation index calculation unit 143 also outputs the noise-removed data D3 in time series in a form in which the noise-removed data D3 is associated with the evaluation index calculated for that noise-removed data D3.

[0068] The processor 140 executes the character recognition program 131 shown in FIG. 2, thereby also functioning as a selection unit 144 that selects noise-removed data D3 for which a relatively high evaluation index has been calculated.

[0069] For each image data group D4 consisting of noise-removed data D3 of a number of frames corresponding to a predetermined character recognition cycle, which is a time interval longer than the time interval between pieces of noise-removed data D3 arranged in time series, the selection unit 144 selects and outputs noise-removed data D3 (hereinafter referred to as high-quality image data D5) for which an evaluation index representing the best image quality in the image data group D4 has been calculated by the evaluation index calculation unit 143. In other words, the selection unit 144 outputs high-quality image data D5 for each character recognition cycle.

[0070] The frame rate of the noise-removed data D3 output from the evaluation index calculation unit 143 is, for example, about 30 fps. That is, the time interval between the noise-removed data D3 arranged in time series is, for example, about 33.3 milliseconds. The character recognition period is, for example, about 500 milliseconds.

[0071] 2, the processor 140 also functions as a ruled line removal unit 145 that performs ruled line removal processing on each piece of high-quality image data D5 to remove components representing ruled lines from that piece of high-quality image data D5. The ruled line removal processing will be described in detail below.

[0072] As shown in Fig. 6(A), the high-quality image data D5 includes components representing ruled lines RL extending vertically and horizontally. That is, the display screen DS of the tidal current meter MA shown in Fig. 1 displays the ruled lines RL together with characters representing the measurement results. However, there is a concern that the ruled lines RL may be mistaken for the characters themselves or part of the characters when recognizing the characters. For example, there is a concern that the vertical ruled lines RL may be mistaken for the number "1."

[0073] Therefore, the ruled line removal section 145 performs ruled line removal processing to remove the components representing the ruled lines RL from the high-quality image data D5. To do this, first, the components representing the ruled lines RL are extracted from the high-quality image data D5.

[0074] Specifically, vertically extending linear regions can be extracted by applying morphological transformation to the high-quality image data D5, which expands and contracts the image data in the vertical direction. Furthermore, by leaving only those vertically extending linear regions that extend continuously for a length greater than the vertical dimension of the characters, vertical ruled lines RL can be extracted.

[0075] Similarly, horizontally extending linear regions can be extracted by applying morphological transformation to the high-quality image data D5, which expands and contracts the image horizontally. Furthermore, horizontal ruled lines RL can be extracted by leaving only those horizontally extending linear regions that extend continuously for a length greater than the horizontal dimension of the characters.

[0076] Fig. 6(B) shows an example of components representing the ruled lines RL extracted from the high-quality image data D5. Note that in Fig. 6(B), for the sake of convenience, the components representing the ruled lines RL are shown inverted black and white to clearly show the ruled lines RL.

[0077] The ruled line removal unit 145 subtracts the components representing the ruled lines RL shown in Figure 6(B), with black and white inverted, from the high-quality image data D5 shown in Figure 6(A). Here, "subtraction" means subtracting the values ​​of corresponding pixels. This completes the ruled line removal process, which removes the components representing the ruled lines RL from the high-quality image data D5.

[0078] 6C shows high-quality image data D5 that has been subjected to the ruled line removal process described above. Hereinafter, high-quality image data D5 that has been subjected to ruled line removal process in this manner will be referred to as ruled line-removed data D6.

[0079] Returning to Figure 3, the explanation will continue. By executing the character recognition program 131 shown in Figure 2, the processor 140 also functions as a character recognition unit 146 that performs character recognition processing to recognize characters from the above-mentioned ruled line-removed data D6. The "characters" referred to here refer to the characters displayed as measurement results on the display screen DS of the tidal current meter MA shown in Figure 1. Note that a known algorithm can be used for the character recognition processing.

[0080] Character recognition unit 146 uses template data 132 in the character recognition process. Template data 132 is data that associates multiple locations where characters to be recognized are located in ruled line removed data D6, which is the target of character recognition processing, with the physical meaning and unit of the characters read from each location.

[0081] Here, "the location where the characters to be recognized are located" refers to an area in the image, and is specified by coordinates in the image. Also, examples of combinations of "physical meaning and unit" include flow speed and knots, direction and degrees, water temperature and degrees, etc.

[0082] By using the template data 132, the character recognition unit 146 can recognize a plurality of numerical values ​​as characters from the ruled line removal processed data D6 in association with their respective physical meanings and units.

[0083] Character recognition unit 146 acquires ruled line removal processed data D6 from ruled line removal unit 145, and each time it performs character recognition processing on that ruled line removal processed data D6 as the recognition target, it outputs character data D7 representing the characters recognized in that character recognition processing. Character data D7 includes multiple numerical values ​​recognized from ruled line removal processed data D6 and associations between each of the multiple numerical values ​​and their physical meanings and units.

[0084] Note that character recognition unit 146 acquires ruled line removal processed data D6 from ruled line removal unit 145 for each of the above-described character recognition cycles. Therefore, character recognition unit 146 repeats the character recognition process in the above-described character recognition cycle. In order to reduce waiting time of character recognition unit 146 and operate character recognition unit 146 efficiently, the character recognition cycle is predetermined as a period approximately equal to the period required for character recognition unit 146 to perform one character recognition process.

[0085] In addition, by executing the character recognition program 131 shown in Figure 2, the processor 140 also functions as an output unit 147 that uses all character data D7 output from the character recognition unit 146 to determine the final character reading result.

[0086] Output unit 147 performs majority voting to determine the final character reading result under the condition that the result of the character recognition process with the highest appearance frequency is adopted, using all character data D7 output from character recognition unit 146. Note that this majority voting process is performed for each location where the character to be recognized is located, represented by the above-mentioned template data 132.

[0087] Then, the output unit 147 outputs the read result character data D8 representing the read result determined by the majority vote process. The communication device 150 acquires the read result character data D8 from the output unit 147, and, as described above, transmits the read result character data D8 to the management server 200 at the timing when communication with the management server 200 shown in FIG. 1 is established.

[0088] In addition, by executing the character recognition program 131 shown in Figure 2, the processor 140 also functions as an overlay display control unit 148 that displays the read result character data D8 output by the output unit 147 on the display device 120.

[0089] As described above, when capturing an image of the display screen DS of the tidal current meter MA shown in Fig. 1, the display device 120 displays an imaging area including the display screen DS so that the captured content can be confirmed. The overlay display control unit 148 causes the display device 120 to display the read result character data D8 overlaid on the imaging area.

[0090] 7 shows an example of the display screen of display device 120 on which read result character data D8 is overlaid. The read result character data D8 representing the characters read from the display screen DS of tidal current meter MA is overlaid on the imaging area including the display screen DS. This allows the user to easily check on-site whether the characters have been correctly read by character recognition device 100.

[0091] Continuing the explanation, returning to Fig. 3, by executing the character recognition program 131 shown in Fig. 2, the processor 140 also functions as an evidence image data identification unit 149 that identifies evidence image data 133 as one sample of the ruled line removal processed data D6 that has been subjected to the character recognition process by the character recognition unit 146.

[0092] Of all the noise-removed data D3 arranged in chronological order, the evidence image data identification unit 149 identifies the ruled line-removed data D6 for which the evaluation index indicating the best image quality has been calculated by the evaluation index calculation unit 143 and which has been subjected to ruled line removal processing by the ruled line removal unit 145 as evidence image data 133. Then, the evidence image data identification unit 149 controls to store the identified evidence image data 133 in the storage device 130.

[0093] The character reading process performed by the character recognition device 100 will be specifically described below with reference to FIG.

[0094] First, on the ship FS, a user starts capturing an image of the display screen DS of the tidal current meter MA as a moving image using the imaging device 110 of the character recognition device 100 (step S1). As a result, the imaging device 110 starts outputting raw image data D1 in chronological order as the captured image.

[0095] Next, the preprocessing unit 141 performs preprocessing on each of the raw image data D1 acquired from the imaging device 110 (step S2). As described above, the preprocessing refers to the cropping process for removing the peripheral region PD shown in Fig. 4(A) and the homography process for approximating the cropped raw image data D1 to a form obtained by capturing the display screen DS shown in Fig. 1 from directly in front.

[0096] Next, the noise removal unit 142 performs noise removal processing on the preprocessed data D2, which is the preprocessed raw image data D1, to remove noise components representing external light reflected on the display screen DS shown in Figure 1 (step S3).

[0097] Next, the evaluation index calculation unit 143 calculates an evaluation index that indicates the degree of quality of image quality for each piece of noise-removed data D3, which is pre-processed data D2 that has been subjected to noise removal processing (step S4).

[0098] Next, the selection unit 144 determines whether or not the calculation of the evaluation index for the noise-removed data D3 for the number of frames corresponding to the character recognition cycle has been completed (step S5). If the calculation of the evaluation index for the noise-removed data D3 for the character recognition cycle has not been completed yet (step S5; NO), the process returns to step S5 again.

[0099] On the other hand, once the calculation of the evaluation index for the noise-removed data D3 for the character recognition cycle has been completed (step S5; YES), the selection unit 144 determines the high-quality image data D5, which is the noise-removed data D3 for which the evaluation index representing the best image quality has been calculated among the noise-removed data D3 for the character recognition cycle (step S6).

[0100] Next, the ruled line removal unit 145 performs ruled line removal processing on the determined high-quality image data D5 to remove components representing ruled lines RL (step S7).

[0101] Next, the character recognition unit 146 performs character recognition processing to recognize characters from the ruled line removal processed data D6, which is the high-quality image data D5 that has been subjected to the ruled line removal processing (step S8). The character recognition unit 146 performs character recognition processing using the template data 132, thereby recognizing a plurality of numerical values ​​as characters from the ruled line removal processed data D6, and associating them with their respective physical meanings and units. The recognition result is output as character data D7.

[0102] Next, the selection unit 144 determines whether or not the calculation of the evaluation index has been completed for all frames of the noise-removed data D3 representing a series of moving images (step S9). If the calculation of the evaluation index has not been completed for all frames of the noise-removed data D3 (step S9; NO), the process returns to step S5.

[0103] On the other hand, if the calculation of the evaluation indexes has been completed for all the noise-removed data D3 (step S9; YES), the output unit 147 performs a majority vote process (step S10) to determine the final character reading result using all the character data D7 output from the character recognition unit 146. The result of the majority vote process is output as read result character data D8 representing the final character reading result.

[0104] Next, the overlay display control unit 148 causes the read result character data D8 output by the output unit 147 to be overlaid and displayed on the display device 120 (step S11). That is, the overlay display control unit 148 causes the read result character data D8 to be displayed on the display device 120, superimposed on an imaging area showing the content imaged by the imaging device 110.

[0105] Next, the evidence image data identification unit 149 stores evidence image data 133 as a sample of the ruled line removal processed data D6 that has been subjected to character recognition processing by the character recognition unit 146 in the storage device 130 (step S12). As described above, in this embodiment, the ruled line removal processed data D6 for which the evaluation index representing the best image quality has been calculated by the evaluation index calculation unit 143 is set as the evidence image data 133.

[0106] According to the character recognition device 100 according to the present embodiment described above, the following effects can be obtained.

[0107] Of the image data group D4 made up of noise-removed data D3 that has been subjected to noise removal processing by the noise removal unit 142, only the high-quality image data D5, for which an evaluation index representing the best image quality has been calculated, is subjected to character recognition processing. This makes it possible to avoid a situation in which image data with relatively poor image quality is subjected to character recognition processing. In other words, it is possible to avoid unnecessary character recognition processing being performed on image data with relatively poor image quality.

[0108] Furthermore, by setting the character recognition period, defined as the time it takes for one set of image data group D4 to be approximately the same as or shorter than the repetition period of the character recognition process, it is possible to eliminate or almost eliminate waiting time in the character recognition process. Therefore, even though high-quality image data D5 with relatively good image quality is selected, the processing time required from the start of imaging to obtain the final read result character data D8 is unlikely to be long.

[0109] Furthermore, the raw image data D1 is not directly subjected to character recognition processing, but rather the raw image data D1 is subjected to preprocessing, noise removal processing, and ruled line removal processing to produce ruled line-removed data D6, which is then subjected to character recognition processing. This reduces the probability of character recognition errors in the character recognition processing, thereby increasing the reliability of character recognition.

[0110] Furthermore, the read result character data D8 is overlaid on the display device 120, allowing the user to check on-site whether the character read process shown in Fig. 8 has been performed correctly. If the character read process has not been performed correctly, the user can redo the character read process to obtain a correct read result.

[0111] Furthermore, the data D6 after the ruled line removal process that has the best image quality is automatically saved in the storage device 130 as evidence image data 133. This evidence image data 133 can be used, for example, to verify whether the character recognition process has been carried out normally or to improve the algorithm of the character recognition process.

[0112] The embodiment has been described above, but the following modifications are also possible.

[0113] FIG. 1 illustrates a configuration for reading characters from the display screen DS of a tidal current meter MA, but the character recognition device 100 can also read characters from the display screens of various devices other than the tidal current meter MA, such as measuring instruments, computers, and displays.

[0114] 2 can be installed on an existing smartphone, tablet, or other computer to enable the computer to realize the functions of the character recognition device 100. The character recognition program 131 can be distributed via a communication line or stored on a recording medium and distributed. [Explanation of symbols]

[0115] 100...Character recognition device, 110...imaging device, 120...display device, 130...storage device, 131...character recognition program, 132...Template data, 133...Evidence image data, 140...processor, 141...pre-processing section, 142...Noise removal unit, 143...Evaluation index calculation unit, 144...Selection Department, 145...crease removal unit, 146...character recognition section, 147...output section, 148...Overlay display control unit, 149...Evidence image data identification unit, 150...Communication devices, 200...Administration server, 300...Fisheries Management Support System, CT: boundary marking element, D1: Raw image data, D2: Preprocessed data (original image data), D3: Noise-removed data (image data), D4: Image data group D5...High-quality image data, D6: Data with border removal processing, D7: Character data, D8: Read result character data, DS…display screen, FS...Ship, MA... tidal current meter (measuring instrument), NE...communication line, PD: peripheral area, RL...ruled line.

Claims

1. an evaluation index calculation unit that acquires a plurality of pieces of image data arranged in time series, which are generated by repeatedly capturing images of a display screen displaying characters, and calculates an evaluation index representing the quality of the image data for each of the acquired image data; a selection unit that selects, for each image data group consisting of image data corresponding to a character recognition cycle that is a predetermined time interval longer than the time interval between the image data arranged in time series, high-quality image data, which is the image data for which the evaluation index that represents the best image quality in the image data group is calculated by the evaluation index calculation unit; a character recognition unit that performs character recognition processing to recognize characters from each of the high-quality image data selected by the selection unit for each of the image data groups; Equipped with the selection unit selects the high-quality image data for each character recognition cycle; the character recognition unit recognizes the characters from the high-quality image data each time the selection unit selects the high-quality image data; the character recognition period is equal to or shorter than the period required for the character recognition unit to recognize the character from one of the high-quality image data; Character recognition device.

2. a noise removal unit that acquires a plurality of original image data arranged in time series, which are the source of the plurality of image data arranged in time series, and performs a noise removal process on each of the acquired original image data to remove noise components representing external light reflected on the display screen; Furthermore, The image data that is used to calculate the evaluation index by the evaluation index calculation unit is noise-removed data obtained by the noise removal unit performing the noise removal process on the original image data. The character recognition device according to claim 1 .

3. The noise removal process A process of creating blurred data by performing blurring on the original image data; A process of subtracting the blurred data from the original image data; The character recognition device of claim 2 , comprising:

4. a pre-processing unit that acquires a plurality of pieces of raw image data arranged in time series from an imaging device that repeatedly captures images of the display screen, and performs a cropping process on each of the acquired raw image data to remove peripheral areas other than the display screen, and a homography process on the raw image data that has been subjected to the cropping process to approximate a form in which the display screen is captured from directly in front of the image; Furthermore, the original image data to be subjected to the noise removal process by the noise removal unit is preprocessed data obtained by performing the cutout process and the homography process on the raw image data by the preprocessing unit; 4. The character recognition device according to claim 2 or 3.

5. A ruled line is displayed on the display screen together with the characters, a ruled line removal unit that performs a ruled line removal process on each of the high-quality image data to remove components representing the ruled lines from the high-quality image data; Furthermore, the character recognition unit subjects the high-quality image data, from which the ruled line removal process has been performed by the ruled line removal unit, to the character recognition process. The character recognition device according to any one of claims 1 to 4.

6. an output unit that uses the results of the character recognition process for each of the high-quality image data to identify a final reading result of the characters under the condition that the result of the character recognition process with the highest appearance frequency is adopted, and outputs read-result character data representing the identified reading result; The character recognition device according to claim 1 , further comprising:

7. a display device that displays an imaging area including the display screen so that the content of the imaging can be confirmed when imaging the display screen; an overlay display control unit that causes the read result character data output by the output unit to be displayed on the display device by overlaying it on the imaging area; The character recognition device according to claim 6 , further comprising:

8. an evidential image data specification unit that specifies, among all the image data arranged in chronological order, the image data for which the evaluation index indicating the best image quality is calculated by the evaluation index calculation unit as evidential image data, and controls the storage of the specified evidential image data; The character recognition device according to claim 1 , further comprising:

9. Computer, an evaluation index calculation unit that acquires a plurality of pieces of image data arranged in time series, which are generated by repeatedly capturing images of a display screen displaying characters, and calculates an evaluation index representing the quality of the image data for each of the acquired image data; a selection unit that selects, for each image data group consisting of image data corresponding to a character recognition cycle that is a predetermined time interval longer than the time interval between the image data arranged in time series, high-quality image data, which is the image data for which the evaluation index that represents the best image quality in the image data group has been calculated by the evaluation index calculation unit; a character recognition unit that performs character recognition processing to recognize the characters from each of the high-quality image data selected by the selection unit for each of the image data groups; It functions as the selection unit selects the high-quality image data for each character recognition cycle; the character recognition unit recognizes the characters from the high-quality image data each time the selection unit selects the high-quality image data; the character recognition period is equal to or shorter than the period required for the character recognition unit to recognize the character from one of the high-quality image data; Character recognition program.

Images