Text data generation system, text data generation method, text data generation program

Abstract

To enable the generation of text data for text images contained in screens with various configurations. [Solution] The system comprises a first system and a second system, the second system being a system that generates text data of character images from image data of a screen displayed on a display device in the first system. The second system comprises an image acquisition unit that receives a video signal supplied to the display device in the first system from the first system and acquires image data from the video signal, a character recognition area setting unit that sets an area in the image data in which a character image from which text data should be acquired is displayed, and a character recognition unit that performs character recognition processing on the area set by the character recognition area setting unit and generates text data of the character image.

Description

Technical Field

[0004] , , , , , , , , , ,

[0005] , , , , ,

[0001] The present invention relates to a text data generation system, a text data generation method, and a text data generation program for generating text data from character images included in video signals.

Background Art

[0002] When character information such as numerical values is included in the display screen of a certain system, in order to use the character information in another system, there may be a case where it is desired to capture it as text data. In this case, if the source code (such as HTML source code) of the display screen can be obtained, the text data can be extracted from the source code and captured. However, in the case of a system operated by another company or a dedicated system of various devices, it is often impossible to obtain the source code.

[0003] On the other hand, for example, in Patent Document 1, in a medical institution such as a hospital, in order to identify a patient examined by an examination device, an image signal of a screen output from an examination device control device is captured, and the patient ID display column in the captured image is subjected to optical character recognition (OCR) processing, whereby the patient ID is obtained as text data.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] On the other hand, as mentioned above, when dealing with output video from systems operated by other companies or dedicated systems for various devices, the screen configurations vary. However, the system described in Patent Document 1 targets a specific inspection equipment control device, making it difficult to accommodate various screen configurations.

[0007] This invention has been made in view of the above points, and its purpose is to enable the generation of text data for character images contained in screens with various configurations. [Means for solving the problem]

[0008] To solve the above problems, the text data generation system of the present invention is An image acquisition unit that acquires image data of a screen from a video signal of a screen that includes character images, A character recognition area setting unit sets the area in the image data in which character images from which text data should be acquired are displayed. A character recognition unit performs character recognition processing on the area set by the character recognition area setting unit in the image data and generates text data of the character image displayed in the area. It is equipped with.

[0009] In this application, "characters" include kanji, alphanumeric characters, hiragana, katakana, and other symbols listed in the JIS X 0208 code table, as well as alphabets used in various languages. Furthermore, it includes not only single characters but also strings of characters and numerical sequences.

[0010] According to the present invention, image data, which is image data of the screen, is obtained from the video signal, and this image data is subjected to character recognition to generate text data, thus eliminating the need to modify existing systems. Furthermore, since the area of ​​the character image from which text data should be obtained can be set, it is possible to handle various systems with different screen formats simply by setting the area.

[0011] In the present invention, if the character image can change over time, the image acquisition unit may repeatedly acquire the image data at predetermined time intervals, or it may acquire the image data each time any character image displayed in the area set by the character recognition area setting unit changes. In this way, even when the character image changes over time, the text data of the character image can be generated chronologically.

[0012] Furthermore, the image acquisition unit may branch the video signal input to the display device in another system and acquire the image data from this branched video signal. In this way, text data can be generated from character images simply by branching the video signal of another system.

[0013] Furthermore, the video signal may be provided by a server connected to a network, and the image acquisition unit may access the server to acquire the image data. In this way, the video signal from a system located remotely can be acquired and text data can be generated simply by accessing the server. In this case, assuming the network is the internet, it is possible to access a web server that provides various types of information via the internet and generate text data from the screen of that web server.

[0014] Furthermore, in the present invention, the character recognition area setting unit may store character recognition area setting data that identifies the set area in association with the identification information of the image data. In this way, for image data that has been processed in the past, the stored character recognition area setting data can be read, eliminating the need to set the character recognition area again and improving work efficiency.

[0015] The present invention also provides a text data generation system including a first system and a second system, wherein the second system is configured to generate text data of a character image from image data of a screen including the character image displayed on a display device in the first system. The second system includes: an image acquisition unit configured to acquire the image data from a video signal supplied from the first system to a display device in the first system; a character recognition area setting unit configured to set an area in which a character image from which text data is to be acquired is displayed in the image data; and a character recognition unit configured to perform a character recognition process on the area set by the character recognition area setting unit in the acquired image data and generate text data of the character image displayed in the area. The second system is provided with the above components.

Advantages of the Invention

[0016] According to the present invention, text data of character images included in screens with various configurations can be generated.

Brief Description of the Drawings

[0017] [Figure 1] FIG. 23 is a configuration diagram of an information system according to an embodiment of the present invention. [Figure 2] FIG. 26 is a diagram showing an example of an image acquired by an image capture unit. [Figure 3] FIG. 29 is a diagram showing an example of a data structure representing an OCR area set by an OCR area setting unit for the screen shown in FIG. 2. [[ID=,31]] [Figure 4] FIG. 32 is a diagram showing an example of OCR data generated based on the OCR area set as shown in FIG. 3. [Figure 5] FIG. 35 is a flowchart of a process executed by a PC 22 in the present embodiment. [Figure 6] FIG. 38 is a configuration diagram of an information system according to a second embodiment of the present invention.

Modes for Carrying Out the Invention

[0018] FIG. 1 is an overall configuration diagram of a system 1 according to an embodiment of the present invention. As shown in the figure, the system 1 includes a first system 10 and a second system 20. The system 1 is used, for example, in the following situations.

[0019] (1) When an employee of a certain company has a meeting with another company and wants to obtain and utilize the text data of the characters displayed as an image on the first system 10 operated by the other company in the second system 20. (2) When the first system 10 is a dedicated system for various machines (for example, construction machines such as cranes and shield machines), and it is desired to obtain and utilize the text data of the characters displayed as an image on the display screen of the first system 10 in the second system 20.

[0020] If the source code of the screen can be obtained from the first system 10, it is sufficient to extract the text data from the source code. However, it is rare to obtain the source code of the screen from the first system 10, and in most cases, only the image data displayed on the screen of the first system 10 can be obtained. In such a case, in order to obtain the text data of the screen from the first system 10, it is necessary to modify the inside of the first system 1, but this is not a practical measure.

[0021] On the other hand, in the system 1 of the present embodiment, without modifying the first system 10, the second system 20 can generate the text data representing the characters displayed as an image on the first system 10. That is, the second system 20 corresponds to the "text data generation system" of the present invention.

[0022] As shown in FIG. 1, the first system 10 includes a personal computer (PC) 12 and a display device 14 that displays the output video of the PC 12. As will be described later, the output video of the PC 12 includes characters that can change over time, such as real-time measurement data.

[0023] PC12 outputs a video signal from its HDMI® terminal and supplies it to the display device 14. In this embodiment, an HDMI splitter 16 is provided between PC12 and the display device 14. The output video signal from PC12 is supplied to the input terminal of the HDMI splitter 16, and one output signal of the HDMI splitter 16 is supplied to the display device 14, while the other output signal is supplied to the second system 20. In other words, the second system 20 is supplied with the same video signal as the video signal supplied to the display device 14.

[0024] The second system 20 includes a PC 22, a display device 24, an input device 26, and an HDMI / USB converter 28. The HDMI / USB converter 28 converts the video signal from the PC 12 supplied by the HDMI splitter 16 into a USB signal and outputs it.

[0025] The PC22 includes an image capture unit 30, an image acquisition unit 32, an OCR processing unit 34, an OCR area setting unit 36, a data output destination setting unit 38, and a data output unit 40.

[0026] The image capture unit 30 is an external or internal unit of the PC 22. The video signal output by the HDMI / USB converter 28 is input to the USB terminal of the image capture unit 30. The image capture unit 30 has the function of capturing the input video signal and outputting still image data.

[0027] Furthermore, the image acquisition unit 32, OCR processing unit 34, OCR area setting unit 36, data output destination setting unit 38, and data output unit 40 are realized by the CPU of PC 22 executing a program. Note that the OCR processing unit 34 corresponds to the "character recognition unit" of the present invention, and the OCR area setting unit 36 ​​corresponds to the "character recognition area setting unit" of the present invention.

[0028] The image acquisition unit 32 outputs a capture command to the image capture unit 30 and acquires the captured image data from the image capture unit 30 in accordance with the command. As described above, the image data acquired by the image acquisition unit 32 is the image data of the screen displayed on the display device 14 in the first system 10.

[0029] As described above, the character images included in the video output from the PC 11 of the first system 10 may change over time. For this reason, the image acquisition unit 32 repeatedly acquires image data using the image capture unit 30 and associates timestamp information indicating the time of capture with each image data.

[0030] The OCR processing unit 34 generates text data for each character image in the image data acquired by the image acquisition unit 32 by performing OCR processing on the character image contained within the area set by the OCR area setting unit 36 ​​(hereinafter referred to as the OCR area). The text data generated by the OCR processing unit 34 is associated with timestamp information corresponding to each image data.

[0031] Furthermore, when an input device 26, such as a keyboard or mouse, initiates a start operation, the image acquisition unit 32 starts capturing the video signal using the image capture unit 30, and repeatedly captures the signal at predetermined time intervals until an end operation is initiated from the input device 26, thereby acquiring image data. The OCR processing unit 34 may perform OCR processing on image data each time image data is acquired by the image acquisition unit 32, or it may perform OCR processing on multiple acquired image data at once after the acquisition of image data has finished.

[0032] The data output destination setting unit 38 receives user input through the input device 26 to specify the output destination for the text data generated by the OCR processing unit 34. Specifically, it accepts input to select whether to send the text data externally or save it to an internal storage device. If sending externally, it accepts input for the destination address, and if saving to the storage device, it accepts input for the saved file name. The accepted information is then set as output destination information.

[0033] The data output unit 40 transmits the text data generated by the OCR processing unit 34, along with timestamp information, to the specified destination via the network 42, such as a LAN or the Internet, or saves it to a storage device with the specified file name, according to the output destination information set by the data output destination setting unit 38.

[0034] Figure 2 shows an example of a screen displayed on the display device 14 of the first system 10. As described above, the video data of this screen is supplied to the second system 20. The example in Figure 2 shows a case where the first system 10 is a crane monitoring system that acquires and displays various measured values ​​indicating the operating status of a crane at a construction site. In the "Current Value" column of Figure 2, the values ​​of "Actual Load," "Rated Load," "Working Radius," "Slewing Angle," "Hoisting Speed," "Lifting Height," "Instantaneous Wind Speed," "CNV Input Power," "Jib Angle," and "Hoisting INV Output Current" are displayed in real time from top to bottom. Therefore, the values ​​displayed in each of the "Current Value" columns may change moment by moment.

[0035] The image data of the screen shown in Figure 2 is acquired by the image acquisition unit 32 and processed by the OCR processing unit 34. The figure shows that the OCR area setting unit 36 ​​has set areas A1 to A10 as OCR areas in each of the "current value" display fields.

[0036] In other words, before the OCR processing unit 34 processes the screen shown in Figure 2 using OCR, the OCR area setting unit 36 ​​displays the captured screen on the display device 24, and the user sets the OCR area by operating the input device 26 on this screen (for example, by dragging the cursor with a mouse). At that time, the user can set a screen name (for example, "Crane Monitoring Screen") and also set item names for each OCR area (in the example in Figure 2, "Actual Load", "Rated Load", "Working Radius", "Slewing Angle", etc.).

[0037] Figure 3 shows an example of data used to set the OCR area (hereinafter referred to as OCR area setting data) for the screen shown in Figure 2, which is set by the OCR area setting unit 36. As shown in Figure 3, the OCR area setting data includes a screen identifier B1, a screen name B2, an area identifier B3, an area name B4, and area coordinate information (top-left and bottom-right coordinates in the example shown in the figure) B5, which are set for each OCR area. The screen identifier B1 and area identifier B3 are automatically generated by the OCR area setting unit 36.

[0038] The OCR area setting unit 36 ​​saves the OCR area setting data to a storage device. The file name used for saving may be set by the user, or it may be automatically generated based on the screen identifier and screen name. This means that for screens where the OCR area has been set in the past, it is only necessary to read the saved OCR area setting data for that screen, and it is not necessary to set it again.

[0039] The OCR processing unit 34, based on the OCR area setting data, extracts an image of the area specified by the area coordinate information B5 from the image data for each OCR area and passes it to the OCR program, receiving the text data which is the result of the OCR processing. Then, for each OCR area, it associates the received text data with the area name B4 and stores it as OCR data together with the screen identifier B1 and screen name B2.

[0040] Figure 4 shows an example of OCR data generated based on the OCR area setting data shown in Figure 3. As shown in the figure, the OCR data consists of a screen identifier B1, a screen name B2, an area identifier B3 and area name B4 for each OCR area, and text data C1 which is the OCR result.

[0041] Furthermore, the image acquisition unit 32 repeatedly captures the video signal to acquire multiple image data, and OCR processing is performed on each image data. As a result, the same number of data entries consisting of region identifier B3, region name B4, and text data C1 are generated as the number of acquired image data entries. Therefore, as shown in Figure 4, the OCR processing unit 34 attaches the sequential number D1 generated for each image data entry and the timestamp information D2 associated with the image data to each data entry consisting of region identifier B3, region name B4, and text data C1.

[0042] As illustrated in Figure 4, the OCR data is sent to a specified output destination by the data output unit 40, or saved with a specified file name, and used by processing it with an appropriate application program, as described above. In this case, for example, saving the OCR data in CSV format can increase its versatility when using the data.

[0043] Figure 5 is a flowchart of the process performed by PC22 in this embodiment. First, in step S10, the image acquisition unit 32 displays the image captured by the image capture unit 30 on the display device 24. Next, in step S12, the system accepts input indicating whether the displayed screen has been processed before, that is, whether the OCR area setting data has already been saved. As a result, if the OCR area setting data is already saved, in step S14, the file name of the saved OCR area setting data is specified and read from the storage device. On the other hand, if it is not saved, in step S16, the OCR area setting unit 36 ​​sets the OCR area and saves it to the storage device as OCR area setting data.

[0044] Following step S14 or step S16, in step S18, the data output destination setting unit 38 sets the output destination for the OCR data, which is the result of the OCR processing. Next, in step S20, the system waits until the OCR start operation is performed. Once the OCR start operation is performed, in step S22, the image acquisition unit 32 causes the image capture unit 30 to capture the video signal, and the captured image data is acquired.

[0045] Next, in step S24, it is determined whether or not an instruction to terminate the OCR process has been given. If no termination instruction has been given, the system waits for a predetermined time in step S26, then returns to step S22, where the image acquisition unit 32 acquires the image data captured by the image capture unit 30. In other words, the acquisition of image data in step S22 is repeated at predetermined time intervals until an instruction to terminate the OCR process is given in step S24.

[0046] On the other hand, if an instruction to terminate the OCR process is given in step S24, in step S28, the OCR processing unit 34 performs OCR processing on each image data repeatedly acquired in step S22, for each OCR region identified by the OCR region setting data read in step S14 or set in step S16. In this processing flow, OCR processing is performed on repeatedly acquired image data in batches; however, as mentioned above, OCR processing may be performed each time image data is acquired in step S22.

[0047] Next, in step S28, the OCR data, which is the result of the OCR processing, is sent to the output destination set in step S18, or saved to a storage device with the specified file name.

[0048] As described above, according to this embodiment, when the PC 12 of the first system 10 outputs video containing characters as images, the character images contained in the video can be obtained as text data by capturing the video output by the PC 12 and performing OCR processing on it.

[0049] Furthermore, since the output video signal from PC12 only needs to be provided to the second system 20 by installing an HDMI splitter 16 between PC12 and the display device 14, no modification to the first system 10 is necessary.

[0050] Furthermore, the OCR area setting unit 36 ​​allows the area of ​​the character image from which text data should be acquired to be set based on the screen output by the PC 12 of the first system 10. Therefore, regardless of the screen configuration, the necessary text data for the character image can be acquired according to the screen configuration.

[0051] Furthermore, since the OCR processing area data is saved in association with the screen identifier, it is unnecessary to reconfigure the OCR processing area for screens that have been processed in the past, thus saving setup time and improving convenience.

[0052] Next, other embodiments of the present invention will be described. Figure 6 shows a system 101, which is a second embodiment of the present invention. In this embodiment, the same components as those in the first embodiment shown in Figure 1 are omitted or simplified in their description.

[0053] In the system 101 of this embodiment, the PC 12 of the first system 110 is connected to the server 116. The server 116 is connected to a network 200, which is either a LAN or the Internet, and has the function of distributing the output screen of the PC 12 to terminals that access it via the network 200.

[0054] Meanwhile, PC22 of the second system 120 is equipped with a browser unit 130 for accessing server 116 via network 200 and acquiring output video from PC12. The browser unit 130 is implemented by a browser program executed by the CPU of PC22. In this embodiment, the image acquisition unit 32 causes the image capture unit 30 to capture the video acquired by the browser unit 130 as image data.

[0055] In this embodiment, the second system 120 accesses the server 116 of the first system 110 to acquire the output video from the PC 12. Therefore, unlike in the first embodiment, it is not necessary to provide a video signal splitter such as an HDMI splitter 16 between the PC 12 of the first system 110 and the display device 14.

[0056] Furthermore, since the first system 110 and the second system 120 can be connected via the network 200, this method can also be applied when the two systems are located in different locations, such as when the first system 110 is installed at a factory or construction site, and the second system 120 is installed in an office or other location far from the site.

[0057] Furthermore, if the network 200 to which server 116 is connected is the internet, and server 116 functions as a web server, and the first system 110 is a system that makes information widely available to the public (for example, a system for a website that publishes weather information in real time), then it is possible to access server 116 from the second system 120 and obtain text data. In other words, the present invention can also be applied when obtaining text data from a screen published by a web server on the internet.

[0058] In the above embodiment, the image acquisition unit 32, OCR processing unit 34, OCR area setting unit 36, data output destination setting unit 38, and data output unit 40 are implemented by local PCs 12 and 22, but these functions may also be located on the cloud.

[0059] Furthermore, although the above embodiment assumes that the video signal is output as an HDMI signal, the video signal standard is not limited to this, and it is possible to use a video signal of an appropriate standard. In that case, instead of the HDMI splitter 16 or the HDMI / USB converter 28, a splitter or converter that corresponds to the video signal standard can be used. Also, the input to the image capture unit 30 is not limited to USB, and any appropriate data transfer standard can be used. [Explanation of symbols]

[0060] 1 System 10 System 1 12, 22 Personal Computers (PCs) 14 Display device 16 HDMI Splitter 20 System 2 22 PC 24 Display devices 26 Input devices 28 HDMI / USB Converters 30 Image Capture Units 32 Image acquisition unit 34 OCR Processing Unit 36 OCR area setting section 38 Data output destination setting section 40 Data output section 42,200 networks

Claims

1. An image acquisition unit that acquires image data of a screen from a video signal of a screen that includes character images, A character recognition area setting unit sets the area in the image data in which character images from which text data should be acquired are displayed. A character recognition unit performs character recognition processing on the region set by the character recognition region setting unit in the image data and generates text data of the character image displayed in the region. A text data generation system equipped with the following features.

2. The text data generation system according to claim 1, wherein the character image may change over time, and the image acquisition unit repeatedly acquires the image data at predetermined time intervals.

3. The text data generation system according to claim 1, wherein the character image may change over time, and the image acquisition unit acquires the image data each time any character image displayed in the area set by the character recognition area setting unit changes.

4. The text data generation system according to claim 1, wherein the image acquisition unit branches the video signal input to the display device in another system and acquires the image data from the branched video signal.

5. The text data generation system according to claim 1, wherein the video signal is provided by a server connected to a network, and the image acquisition unit accesses the server to acquire the image data.

6. The text data generation system according to claim 5, wherein the network is the Internet.

7. The text data generation system according to claim 1, wherein the character recognition area setting unit stores character recognition area setting data that identifies the set area in association with the identification information of the image data.

8. A text data generation system comprising a first system and a second system, wherein the second system generates text data of character images from image data of a screen including character images displayed on a display device in the first system, The second system described above is In the first system, a video signal is supplied to the display device from the first system, and an image acquisition unit acquires the image data from the video signal, A character recognition area setting unit sets the area in the image data in which character images from which text data should be acquired are displayed. A character recognition unit performs character recognition processing on the area set by the character recognition area setting unit in the acquired image data and generates text data of the character image displayed in the area. A text data generation system equipped with the following features.

9. Image acquisition step: Obtaining image data of the screen from the video signal of the screen which includes character images, A character recognition area setting step in which a character image from which text data should be acquired is displayed in the aforementioned image data, A character recognition step which involves performing character recognition processing on the area set in the character recognition area setting step in the acquired image data and generating text data of the character image displayed in the area, A method for generating text data, comprising the following features.

10. On the computer, Image acquisition step: Obtaining image data of the screen from the video signal of the screen which includes character images, A character recognition area setting step in which a character image from which text data should be acquired is displayed in the aforementioned image data, A character recognition step which involves performing character recognition processing on the area set in the character recognition area setting step in the acquired image data and generating text data of the character image displayed in the area, A text data generation program that executes [this command].

Images