Information processing device and program

Abstract

To provide an information processing apparatus and a program capable of displaying an image in which an operator can easily visually recognize character lines.SOLUTION: According to an embodiment, an information processing apparatus comprises an interface and a processor. The interface transmits and receives data to and from an external apparatus and input and output apparatuses. The processor receives OCR intermediate information generated in the process of OCR processing for an image containing a form on which text lines are written from the external apparatus through the interface; corrects a display method of the form image showing the form based on the OCR intermediate information; causes the input and output apparatus to display the corrected form image through the interface; obtains characters input to the input and output apparatus through the interface; and transmits character information based on the obtained characters to the external apparatus through the interface.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing device and a program. [Background technology]

[0002] There is provided a recognition system that includes a recognition device that reads lines of characters such as addresses using OCR (optical character recognition) processing, and a VCS (video coding system) that accepts character input from an operator if the recognition device fails to recognize the lines of characters.

[0003] The VCS displays an image including lines of text on a screen to an operator and accepts text input by the operator. Conventionally, a VCS may display an image in which the character lines are difficult for an operator to visually recognize depending on the inclination or size of the character lines. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-103163 A Summary of the Invention [Problem to be solved by the invention]

[0005] In order to solve the above problem, an information processing device and a program are provided that can display an image in which a character line is easily visible to an operator. [Means for solving the problem]

[0006] According to an embodiment, an information processing device includes an interface and a processor. The interface transmits and receives data to and from an external device and an input / output device. The processor receives, via the interface, from the external device, OCR intermediate information generated in the course of OCR processing for an image including a form on which text lines are written, corrects a display method of a form image including the form based on the OCR intermediate information, causes the input / output device to display the corrected form image via the interface, obtains characters input to the input / output device via the interface, and transmits character information based on the obtained characters to the external device via the interface. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a recognition system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of the OCR device according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating an example of the configuration of the control device according to the first embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of the configuration of a VCS according to the first embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of a luggage image according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of the configuration of OCR intermediate information according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a form image according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a form image after correction according to the first embodiment. [Figure 9] FIG. 9 is a flowchart illustrating an operation example of the VCS according to the first embodiment. [Figure 10] FIG. 10 is a flowchart illustrating an example of the operation of the control device according to the second embodiment. [Figure 11]FIG. 11 is a flowchart showing an operation example of the VCS according to the second embodiment. [Figure 12] FIG. 12 is a diagram showing an example of a selection screen according to the third embodiment. [Figure 13] FIG. 13 is a flowchart showing an operation example of the VCS according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments will be described with reference to the drawings. (First embodiment) The recognition system according to the embodiment reads destinations such as addresses from packages inserted into a sorting device. The recognition system sets the sorting destination of the package (e.g., a chute of the sorting device) based on the read destination. If the recognition system fails to read the destination, it accepts input of the destination from an operator who visually views the destination.

[0009] Fig. 1 shows an example of the configuration of a recognition system 1 according to an embodiment. As shown in Fig. 1, the recognition system 1 includes a classification device 2, a camera 4, an OCR device 10, a control device 20, and a VCS 30. The OCR device 10 is connected to the camera 4 and the control device 20. The control device 20 is connected to the classification device 2 and the VCS 30.

[0010] Note that the recognition system 1 may further include components as necessary in addition to the components shown in FIG.

[0011] The sorting device 2 sorts luggage inserted by a worker, a conveyor belt, a robot, or the like. The sorting device 2 receives destination information (text information) related to the destination of the luggage from the control device 20. The sorting device 2 sorts the luggage based on the destination information. For example, the sorting device 2 sorts the luggage into a chute, pocket, cart, tray, or the like as a sorting destination. For example, the sorting device 2 is composed of a sorter, a conveyor belt, a robot, or the like.

[0012] The camera 4 photographs the baggage inserted into the sorting device 2. The camera 4 photographs the slip on which the destination is written. For example, the camera 4 photographs the side on which the slip is attached. The camera 4 supplies the image obtained by photographing (baggage image) to the OCR device 10.

[0013] For example, the camera 4 is a CCD (Charge Coupled Device) camera. The camera 4 may also include a light source that illuminates the baggage.

[0014] The OCR device 10 acquires an image of a package from the camera 4. The OCR device 10 generates destination information by OCR processing and transmits the destination information to the control device 20. Furthermore, if the OCR process fails, the OCR device 10 generates OCR intermediate information related to the OCR process. The OCR device 10 transmits the OCR intermediate information and the form image to the control device 20. The OCR device 10 and the OCR intermediate information will be described in detail later.

[0015] The control device 20 (external device) acquires the destination information from the OCR device 10 and transmits it to the sorting device 2 . In addition, the control device 20 acquires OCR intermediate information and package images from the OCR device 10, and transmits them to the VCS 30. In addition, the control device 20 acquires destination information from the VCS 30, and transmits them to the sorting device 2. The control device 20 will be described later.

[0016] The VCS 30 obtains the OCR intermediate information and the form image from the control device 20. The VCS 30 extracts the form image showing the form from the baggage image based on the OCR intermediate information and displays it. The VCS 30 accepts input of the destination from the operator who visually checks the displayed form image. The VCS 30 transmits destination information indicating the input destination to the control device 20. The VCS 30 will be described later.

[0017] Next, the OCR device 10 will be described. Fig. 2 shows an example of the configuration of the OCR device 10 according to the embodiment. Fig. 2 is a block diagram showing an example of the configuration of the OCR device 10. As shown in Fig. 2, the OCR device 10 includes a processor 11, a ROM 12, a RAM 13, an NVM 14, a communication unit 15, an operation unit 16, a display unit 17, a camera interface 18, and the like.

[0018] The processor 11, the ROM 12, the RAM 13, the NVM 14, the communication unit 15, the operation unit 16, the display unit 17, and the camera interface 18 are connected to one another via a data bus or the like. The OCR device 10 may include components as necessary in addition to the components shown in FIG. 2, and certain components may be excluded from the OCR device 10.

[0019] The processor 11 has a function of controlling the overall operation of the OCR device 10. The processor 11 may include an internal cache and various interfaces, etc. The processor 11 realizes various processes by executing programs stored in advance in the internal memory, the ROM 12, or the NVM 14.

[0020] Some of the various functions realized by the processor 11 executing the programs may be realized by hardware circuits. In this case, the processor 11 controls the functions executed by the hardware circuits.

[0021] The ROM 12 is a non-volatile memory that stores in advance a control program, control data, etc. The control program and control data stored in the ROM 12 are incorporated in advance according to the specifications of the OCR device 10.

[0022] The RAM 13 is a volatile memory. The RAM 13 temporarily stores data being processed by the processor 11. The RAM 13 stores various application programs based on instructions from the processor 11. The RAM 13 may also store data necessary for executing the application programs and execution results of the application programs.

[0023] The NVM 14 is a non-volatile memory in which data can be written and rewritten. The NVM 14 is, for example, configured with a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The NVM 14 stores control programs, applications, various data, etc. according to the operational use of the OCR device 10.

[0024] For example, the NVM 14 stores dictionary information for performing OCR processing. For example, the dictionary information indicates the feature amount of each character. The dictionary information may be a network obtained by machine learning. The configuration of the dictionary information is not limited to a specific configuration.

[0025] The communication unit 15 is an interface for transmitting and receiving data to and from the control device 20, etc. For example, the communication unit 15 is an interface that supports a wired or wireless LAN (Local Area Network) connection.

[0026] The operation unit 16 accepts various operation inputs from an operator. The operation unit 16 transmits a signal indicating the input operation to the processor 11. The operation unit 16 may be configured from a touch panel.

[0027] The display unit 17 displays image data from the processor 11. For example, the display unit 17 is configured with a liquid crystal monitor. When the operation unit 16 is configured with a touch panel, the display unit 17 may be formed integrally with the operation unit 16.

[0028] The camera interface 18 is an interface that connects to the camera 4. The camera interface 18 transmits a signal from the processor 11 to the camera 4. The camera interface 18 also acquires a signal (such as a baggage image) from the camera 4 and transmits it to the processor 11.

[0029] Next, the control device 20 will be described. Fig. 3 shows a configuration example of the control device 20 according to the embodiment. Fig. 3 is a block diagram showing a configuration example of the control device 20. As shown in Fig. 3, the control device 20 includes a processor 21, a ROM 22, a RAM 23, an NVM 24, a communication unit 25, an operation unit 26, a display unit 27, and the like.

[0030] The processor 21, the ROM 22, the RAM 23, the NVM 24, the communication unit 25, the operation unit 26, and the display unit 27 are connected to one another via a data bus or the like. The control device 20 may include components as necessary in addition to the components shown in FIG. 3, and certain components may be excluded from the control device 20.

[0031] The processor 21 has a function of controlling the overall operation of the control device 20. The processor 21 may include an internal cache and various interfaces, etc. The processor 21 realizes various processes by executing programs stored in advance in the internal memory, the ROM 22, or the NVM 24.

[0032] Some of the various functions realized by the processor 21 executing the programs may be realized by a hardware circuit. In this case, the processor 21 controls the functions executed by the hardware circuit.

[0033] The ROM 22 is a non-volatile memory in which control programs, control data, etc. are stored in advance. The control programs and control data stored in the ROM 22 are incorporated in advance in accordance with the specifications of the control device 20.

[0034] The RAM 23 is a volatile memory. The RAM 23 temporarily stores data being processed by the processor 21. The RAM 23 stores various application programs based on instructions from the processor 21. The RAM 23 may also store data necessary for executing the application programs and execution results of the application programs.

[0035] The NVM 24 is a non-volatile memory in which data can be written and rewritten. The NVM 24 is, for example, configured with an HDD, an SSD, a flash memory, etc. The NVM 24 stores control programs, applications, various data, etc. according to the operational use of the control device 20.

[0036] The communication unit 25 is an interface for transmitting and receiving data to and from the sorting device 2, the OCR device 10, the VCS 30, etc. For example, the communication unit 25 is an interface that supports a wired or wireless LAN connection.

[0037] The communication unit 25 may be composed of an interface for transmitting and receiving data to and from the sorting device 2, an interface for transmitting and receiving data to and from the OCR device 10, and an interface for transmitting and receiving data to and from the VCS 30.

[0038] The operation unit 26 receives various operation inputs from an operator. The operation unit 26 transmits a signal indicating the input operation to the processor 21. The operation unit 26 may be configured as a touch panel.

[0039] The display unit 27 displays image data from the processor 21. For example, the display unit 27 is configured with a liquid crystal monitor. When the operation unit 26 is configured with a touch panel, the display unit 27 may be formed integrally with the operation unit 26.

[0040] Next, the VCS 30 will be described. Fig. 4 shows an example of the configuration of the VCS 30 according to the embodiment. Fig. 4 is a block diagram showing the example of the configuration of the VCS 30. As shown in Fig. 4, the VCS 30 includes a management device 3 and an input / output device 40. The management device 3 and the input / output device 40 are connected to each other. Here, the VCS 30 includes three input / output devices 40 .

[0041] The management device 3 (information processing device) controls the entire VCS 30. The management device 3 includes a processor 31, a ROM 32, a RAM 33, an NVM 34, a communication unit 35, an input / output interface 36, and the like. The processor 31, the ROM 32, the RAM 33, the NVM 34, the communication unit 35, and the input / output interface 36 are connected to one another via a data bus or the like.

[0042] The processor 31 has a function of controlling the overall operation of the management device 3. The processor 31 may include an internal cache and various interfaces. The processor 31 realizes various processes by executing programs stored in advance in the internal memory, the ROM 32, or the NVM 34.

[0043] Some of the various functions realized by the processor 31 executing the programs may be realized by a hardware circuit. In this case, the processor 31 controls the functions executed by the hardware circuit.

[0044] The ROM 32 is a non-volatile memory in which control programs, control data, etc. are stored in advance. The control programs and control data stored in the ROM 32 are incorporated in advance in accordance with the specifications of the management device 3.

[0045] The RAM 33 is a volatile memory. The RAM 33 temporarily stores data being processed by the processor 31. The RAM 33 stores various application programs based on instructions from the processor 31. The RAM 33 may also store data necessary for executing the application programs and execution results of the application programs.

[0046] The NVM 34 is a non-volatile memory in which data can be written and rewritten. The NVM 34 is, for example, configured with an HDD, an SSD, a flash memory, etc. The NVM 34 stores control programs, applications, various data, etc. according to the operational purpose of the management device 3.

[0047] The communication unit 35 is an interface for transmitting and receiving data to and from the control device 20, etc. For example, the communication unit 35 is an interface that supports a wired or wireless LAN connection.

[0048] The input / output interface 36 is an interface that transmits and receives data to and from the input / output device 40. The input / output interface 36 transmits form images and the like to the input / output device 40. The input / output interface 36 also acquires destinations and the like from the input / output device 40. For example, the input / output interface 36 is an interface that supports wired or wireless LAN connections.

[0049] The communication unit 35 and the input / output interface 36 may be integrally configured.

[0050] The input / output device 40 displays form images and the like from the management device 3. The input / output device 40 also receives an input of a destination from an operator. The input / output device 40 transmits the input destination to the management device 3.

[0051] The input / output device 40 includes an operation unit 41 and a display unit 42 . The operation unit 41 receives various operation inputs from an operator. The operation unit 26 may be configured with a touch panel. The input / output device 40 transmits a signal indicating the operation input to the operation unit 41 to the management device 3.

[0052] The display unit 42 displays image data (such as a form image) from the management device 3. For example, the display unit 27 is configured with a liquid crystal monitor. When the operation unit 26 is configured with a touch panel, the display unit 27 may be formed integrally with the operation unit 26. For example, the input / output device 40 is a desktop PC or a notebook PC.

[0053] Note that the VCS 30 may include components as necessary in addition to the components shown in FIG.

[0054] Next, a description will be given of functions realized by the OCR device 10. The functions realized by the OCR device 10 are realized by the processor 11 executing a program stored in the internal memory, the ROM 12, the NVM 14, or the like.

[0055] First, the processor 11 has a function of acquiring a luggage image. Here, it is assumed that the camera 4 is installed in a position where it can photograph the baggage put into the sorting device 2.

[0056] The processor 11 causes the camera 4 to capture an image of the baggage via the camera interface 18. The processor 11 acquires, via the camera interface 18, from the camera 4, an image of the baggage in which the baggage is captured.

[0057] 5 shows an example of a luggage image 200 acquired by the processor 11 from the camera 4. As shown in FIG. 5, the luggage image 200 includes one side of the luggage 100. Furthermore, the package image 200 includes, on one side of the package 100, documents 101 to 103 attached to the package 100. The documents 101 to 103 contain lines of text such as the destination.

[0058] The processor 11 also has a function of acquiring destination information from the package image by OCR processing.

[0059] First, the processor 11 extracts an area in which a form is captured (a form area) from the luggage image 200 using a predetermined image processing. For example, the processor 11 extracts the form area by detecting edges of the form. The processor 11 may also extract the form area using artificial intelligence such as a neural network. The method by which the processor 11 extracts the form area from the luggage image 200 is not limited to a specific method.

[0060] Furthermore, the processor 11 acquires parameters of the extracted form region. Here, the processor 11 acquires the coordinates (for example, center coordinates), size, and angle (tilt of the form region) of the form region.

[0061] When the form area is extracted, the processor 11 extracts an area (text line area) in which a text line (for example, one text line) is captured from the form area. Here, the processor 11 extracts a rectangular text line area (the smallest rectangular area including a text line).

[0062] The processor 11 extracts the destination character line area from the form area using a predetermined image processing. For example, the processor 11 detects and extracts the character line area by pattern recognition. The processor 11 may also extract the character line area using artificial intelligence such as a neural network. The method by which the processor 11 extracts the character line area from the form area is not limited to a specific method.

[0063] Processor 11 also acquires parameters of the extracted character line area. Here, processor 11 acquires the coordinates, size, and angle (slope of the character line area) of the character line area. Processor 11 also acquires a flag indicating whether the character line included in the character line area is handwritten or printed. For example, processor 11 uses a predetermined image processing to determine whether the character line included in the character line area is handwritten or printed.

[0064] When the character line region is extracted, the processor 11 recognizes the character lines contained in the character line region.

[0065] For example, processor 11 extracts an area including one character (character area) from a character line area. After extracting the character area, processor 11 matches the image of the character area with dictionary information. Processor 11 recognizes the character written in each character area through the matching process.

[0066] When the processor 11 recognizes the characters written in each character area, the processor 11 recognizes the destinations written on the forms 101 to 103 based on the recognized characters. When the processor 11 recognizes the destinations, the processor 11 generates destination information based on the recognized destinations.

[0067] When the destination information is generated, the processor 11 transmits the generated destination information to the control device 20 through the communication unit 15.

[0068] In addition, the processor 11 has a function of transmitting the OCR intermediate information and the package image to the control device 20 when the processor 11 fails to acquire the destination information.

[0069] For example, if processor 11 fails to recognize a character, it generates intermediate OCR information, which is made up of information generated during the OCR process.

[0070] Fig. 6 shows an example of the configuration of the OCR intermediate information. As shown in Fig. 6, the OCR intermediate information is composed of baggage image information, form information, character line information, candidate information, and the like.

[0071] The luggage image information is related to the luggage image. Here, the luggage image information is composed of "image size." "Image size" is the size of the luggage image.

[0072] The form information is related to the form. Here, the form information is composed of "form coordinates," "form size," and "form angle."

[0073] "Coordinates of the form" are the coordinates of the form area in the package image. "Size of the form" is the size of the form area in the package image. "Form angle" is the angle of the form area in the package image.

[0074] The character line information is related to the character line. Here, the character line information is composed of "character line coordinates", "character line size", "character line angle" and "handwriting / printing judgment".

[0075] "Text line coordinates" are the coordinates of the text line area in the baggage image. "Text line size" is the size of the text line area in the baggage image. "Text line angle" is the angle of the text line area in the luggage image. "Handwritten / printed determination" indicates whether the character lines in the character line area are handwritten or printed.

[0076] The candidate information relates to a candidate destination. Here, the candidate information is composed of "candidate addresses." "Candidate addresses" are candidates for addresses (destinations) written on a form. For example, the processor 11 generates candidate information including character lines that cannot be confirmed but are likely to be destinations as "candidate addresses."

[0077] When there are multiple form areas, the OCR intermediate information may store form information, character line information, and candidate information for each form. The configuration of the OCR intermediate information is not limited to a specific configuration.

[0078] Furthermore, if the processor 11 fails to acquire any piece of information constituting the OCR intermediate information during the OCR process, the processor 11 generates OCR intermediate information that does not include that piece of information.

[0079] When the OCR intermediate information is generated, the processor 11 transmits the OCR intermediate information and the baggage image to the control device 20 via the communication unit 15.

[0080] Next, a description will be given of functions realized by the control device 20. The functions realized by the control device 20 are realized by the processor 21 executing a program stored in the internal memory, the ROM 22, the NVM 24, or the like.

[0081] The processor 21 has a function of transmitting destination information to the sorting device 2 . When receiving the destination information from the OCR device 10, the processor 21 transmits the received destination information to the sorting device 2 through the communication unit 25.

[0082] In addition, when the processor 21 receives the OCR intermediate information and the luggage image from the OCR device 10, the processor 21 transmits the OCR intermediate information and the luggage image to the VCS 30 via the communication unit 25.

[0083] As described later, the VCS 30 transmits to the control device 20 destination information indicating the destination written on the slip of the package image.

[0084] The processor 21 receives the destination information from the VCS 30 through the communication unit 25. When the processor 21 receives the destination information from the VCS 30, the processor 21 transmits the received destination information to the sorting device 2 through the communication unit 25.

[0085] Next, a description will be given of functions realized by the VCS 30. The functions realized by the VCS 30 are realized by the processor 21 executing a program stored in the internal memory, the ROM 22, the NVM 24, or the like.

[0086] First, the processor 31 has a function of extracting an image in which a form is shown (a form image) from the baggage image.

[0087] Here, it is assumed that the processor 31 receives the OCR intermediate information and the luggage image from the control device 20 via the communication unit 35.

[0088] When the OCR intermediate information and the luggage image are received, the processor 31 determines whether the OCR intermediate information includes form information. If it is determined that the OCR intermediate information includes form information, the processor 31 sets a form area in the luggage image based on the form information in the OCR intermediate information. When the form area is set, the processor 31 obtains an image inside the form area as a form image.

[0089] Fig. 7 shows an example of a form image extracted by the processor 31. In the example shown in Fig. 7, the processor 31 extracts form images 201 to 203. The form images 201 to 203 include forms 101 to 103, respectively. Furthermore, character line regions 301 to 303 are set in the form images 201 to 203.

[0090] The processor 31 also has a function of correcting the form images 201 to 203 based on the OCR intermediate information.

[0091] The processor 31 generates an image in which the display method of the form image is corrected so as to improve the visibility of the operator. Here, the processor 31 generates an image in which the size and angle of the form image are corrected.

[0092] For example, the processor 31 obtains the height of a character line area in a form image based on the character line information of the OCR intermediate information. For example, the processor 11 obtains the height of a character line area based on the “size of a character line” or the like.

[0093] When the height of the character line area is acquired, the processor 31 calculates a magnification for enlarging or reducing the form image so that the height of the character line area becomes a predetermined value (for example, the height of the character line that is the reference for screen display). After calculating the magnification, the processor 31 corrects the size of the form image according to the calculated magnification.

[0094] After correcting the size of the form image, processor 31 corrects the angle of the form image based on the "text line angle." Here, processor 31 corrects the inclination (angle) of the form image so that one side (e.g., the long side) of the text line area has a predetermined inclination, for example, becomes horizontal. Processor 31 may also correct the angle of the form image based on the relative angle between the "text line angle" and the "form angle."

[0095] Moreover, the processor 31 may correct the angle of the form image and then correct the size of the form image.

[0096] Fig. 8 shows an example of a form image corrected by the processor 31. In the example shown in Fig. 8, the processor 31 corrects the form images 201 to 203, respectively, to generate corrected form images 201' to 203'.

[0097] The heights of the text line areas in the corrected form images 201' to 203' are all set to a predetermined value. Also, the corrected form image 201' is corrected so that the long side of the text line area has a predetermined inclination, for example, becomes horizontal.

[0098] When multiple text line areas are set in one form image, the processor 31 may obtain the average or minimum value of the heights of the multiple text line areas as the height of the text line area. In this case, the processor 31 may correct the angle of the form image based on the average value of the angles of the multiple text line areas.

[0099] The processor 31 also has a function of displaying the corrected form image and receiving destination input from the operator.

[0100] After correcting the form image, the processor 31 causes one of the input / output devices 40 to display the corrected form image through the input / output interface 36. That is, the processor 31 causes the display unit 42 of the input / output device 40 to display the corrected form image.

[0101] When the display unit 42 displays the corrected form image, the operator visually checks the corrected form image and inputs the address written on the form into the operation unit 41 of the input / output device 40 .

[0102] The processor 31 acquires the destination input to the operation unit 41 via the input / output interface 36 .

[0103] When the destination is acquired, the processor 31 generates destination information indicating the acquired destination. When the destination information is generated, the processor 31 transmits the generated destination information to the control device 20 through the communication unit .

[0104] When the OCR intermediate information does not include form information, the processor 31 causes one of the input / output devices 40 to display a package image via the input / output interface 36. Here, the operator visually checks the package image and inputs the destination written on the form into the operation unit 41 of the input / output device 40.

[0105] The processor 31 acquires a destination input to the operation unit 41 through the input / output interface 36. When the destination is acquired, the processor 31 generates destination information indicating the acquired destination. When the destination information is generated, the processor 31 transmits the generated destination information to the control device 20 through the communication unit 35.

[0106] Next, an example of the operation of the VCS 30 will be described. FIG. 9 is a flowchart for explaining an example of the operation of the VCS 30.

[0107] First, the processor 31 of the VCS 30 receives the OCR intermediate information and the luggage image from the control device 20 through the communication unit 35 (S11). When the OCR intermediate information and the luggage image are received from the control device 20, the processor 31 determines whether the OCR intermediate information includes form information (S12).

[0108] If it is determined that the OCR intermediate information includes form information (S12, YES), the processor 31 extracts a form image from the package image based on the form information included in the OCR intermediate information (S13). After extracting the form image, the processor 31 corrects the size and inclination of the form image based on the OCR intermediate information (S14).

[0109] After correcting the size and inclination of the form image, the processor 31 causes one of the input / output devices 40 to display the corrected form image through the input / output interface 36 (S15). After causing one of the input / output devices 40 to display the corrected form image, the processor 31 receives input of a destination from an operator through the input / output interface 36 (S16).

[0110] When it is determined that the OCR intermediate information does not include form information (S12, NO), the processor 31 causes one of the input / output devices 40 to display a package image via the input / output interface 36 (S17). After causing one of the input / output devices 40 to display the package image, the processor 31 accepts input of a destination from an operator via the input / output interface 36 (S18).

[0111] When the processor 31 receives input of a destination from the operator (S16 or S18), the processor 31 transmits destination information indicating the destination input in S16 or S18 to the control device 20 via the communication unit 35 (S19). After transmitting the destination information to the control device 20, the processor 31 ends the operation.

[0112] The processor 31 may correct the form image based on the form information. For example, the processor 31 may correct the size and angle of the form image based on the "form size" and "form angle" indicated by the form information. After correcting the size and angle of the form image based on the form information, the processor 31 may further correct the size and angle of the form image based on character line information.

[0113] The OCR intermediate information may also include character line information indicating the vertex coordinates of the character line region. In this case, the processor 31 may calculate the height and angle of the character line region from the vertex coordinates of the character line region.

[0114] The recognition system 1 may also include a plurality of cameras 4 and a plurality of OCR devices 10. The recognition system 1 may also include a plurality of VCSs 30.

[0115] The recognition system configured as above transmits the intermediate OCR information generated during the OCR process to the VCS. Based on the intermediate OCR information, the recognition system corrects the form image so that it is easier for the operator to view. As a result, the recognition system can display an image in which the operator can easily view the character lines. Second Embodiment Next, a second embodiment will be described. The recognition system 1 according to the second embodiment differs from that according to the first embodiment in that the control device 20 extracts a form image from a baggage image. Therefore, the other points are given the same reference numerals and detailed description is omitted.

[0116] The configuration of the recognition system 1 according to the second embodiment is similar to that of the first embodiment, and therefore a description thereof will be omitted.

[0117] Next, a description will be given of functions realized by the control device 20. The functions realized by the control device 20 are realized by the processor 21 executing a program stored in the internal memory, the ROM 22, the NVM 24, or the like. The functions realized by the control device 20 include the following functions in addition to the functions realized by the control device 20 according to the first embodiment.

[0118] The processor 21 has a function of transmitting the OCR intermediate information and the form image to the VCS 30 . As described above, the processor 21 receives the OCR intermediate information and the luggage image from the OCR device 10 through the communication unit 25. Upon receiving the OCR intermediate information and the luggage image, the processor 21 determines whether the OCR intermediate information includes form information.

[0119] When it is determined that the OCR intermediate information includes form information, the processor 21 sets a form area in the package image based on the form information in the OCR intermediate information. When the form area is set, the processor 31 acquires an image inside the form area as a form image.

[0120] When the form image is acquired, the processor 21 transmits the OCR intermediate information and the form image to the VCS 30 via the communication unit 25.

[0121] If it is determined that the OCR intermediate information does not include form information, the processor 21 transmits the package image to the VCS 30 via the communication unit 25. In this case, the processor 21 does not need to transmit the OCR intermediate information to the VCS 30.

[0122] Next, a description will be given of functions realized by the VCS 30. The functions realized by the VCS 30 are realized by the processor 21 executing a program stored in the internal memory, the ROM 22, the NVM 24, or the like. The functions realized by the VCS 30 include the following functions in addition to the functions realized by the VCS 30 according to the first embodiment.

[0123] The processor 31 has a function of transmitting destination information to the control device 20 in response to data from the control device 20 .

[0124] The processor 31 receives data from the control device 20 via the communication unit 35. When the data includes the OCR intermediate information and the form image, the processor 31 corrects the form image based on the OCR intermediate information. The operation example of the processor 31 correcting the form image is similar to that of the first embodiment, and therefore the description is omitted.

[0125] After correcting the form image, the processor 31 displays the corrected form image and accepts an input of a destination from an operator, as in the first embodiment. Upon accepting the input of the destination, the processor 31 transmits destination information indicating the input destination to the control device 20 via the communication unit 35.

[0126] Furthermore, when the data includes a package image, the processor 31 displays the package image and accepts input of a destination from an operator, as in the first embodiment. Upon accepting the input of the destination, the processor 31 transmits destination information indicating the input destination to the control device 20 via the communication unit 35.

[0127] Next, an example of the operation of the control device 20 will be described. FIG. 10 is a flowchart for explaining an example of the operation of the control device 20. As shown in FIG.

[0128] First, the processor 21 of the control device 20 receives the OCR intermediate information and the luggage image from the OCR device 10 through the communication unit 25 (S21). Upon receiving the OCR intermediate information and the luggage image from the OCR device 10, the processor 21 determines whether the OCR intermediate information includes form information (S22).

[0129] When it is determined that the OCR intermediate information includes the form information (S22, YES), the processor 21 extracts the form image from the package image based on the form information included in the OCR intermediate information (S23). After extracting the form image, the processor 21 transmits the OCR intermediate information and the form image to the VCS 30 via the communication unit 25 (S24).

[0130] When it is determined that the OCR intermediate information does not include form information (S22, NO), the processor 21 transmits the package image to the VCS 30 through the communication unit 25 (S25).

[0131] When the OCR intermediate information and the form image are transmitted to the VCS 30 (S24), or when the package image is transmitted to the VCS 30 (S25), the processor 21 waits until the destination information is received from the VCS 30 via the communication unit 25.

[0132] When the destination information is received from the VCS 30 (S26), the processor 21 transmits the destination information to the sorting device 2 via the communication unit 25 (S27). After transmitting the destination information to the sorting device 2, the processor 21 ends its operation.

[0133] Next, an example of the operation of the VCS 30 will be described. FIG. 11 is a flowchart for explaining an example of the operation of the VCS 30.

[0134] First, the processor 31 of the VCS 30 receives data from the control device 20 via the communication unit 35 (S31). When the data is received from the control device 20, the processor 31 determines whether the data includes OCR intermediate information and form information (S32).

[0135] If it is determined that the data includes OCR intermediate information and form information (S32, YES), the processor 31 corrects the size and inclination of the form image based on the OCR intermediate information (S33).

[0136] After correcting the size and inclination of the form image, the processor 31 causes one of the input / output devices 40 to display the corrected form image through the input / output interface 36 (S34). After causing one of the input / output devices 40 to display the corrected form image, the processor 31 receives input of a destination from an operator through the input / output interface 36 (S35).

[0137] When it is determined that the data does not include OCR intermediate information and form information (contains a luggage image) (S32, NO), the processor 31 causes one of the input / output devices 40 to display the luggage image via the input / output interface 36 (S36). After causing one of the input / output devices 40 to display the luggage image, the processor 31 accepts input of a destination from an operator via the input / output interface 36 (S37).

[0138] When the processor 31 receives input of the destination from the operator (S35 or S37), the processor 31 transmits destination information indicating the destination input in S35 or S37 to the control device 20 via the communication unit 35 (S38). After transmitting the destination information to the control device 20, the processor 31 ends the operation.

[0139] In the recognition system configured as above, the control device extracts the form image and transmits it to the VCS, which allows the recognition system to reduce the amount of communication between the control device and the VCS. (Third embodiment) Next, a third embodiment will be described. The recognition system 1 according to the third embodiment differs from that of the first embodiment in that the operator is prompted to select a form for inputting a destination. Therefore, the other points are given the same reference numerals and detailed description is omitted.

[0140] The configuration of the recognition system 1 according to the third embodiment is similar to that of the first embodiment, and therefore a description thereof will be omitted.

[0141] Next, a description will be given of functions realized by the VCS 30. The functions realized by the VCS 30 are realized by the processor 21 executing a program stored in the internal memory, the ROM 22, the NVM 24, or the like. The functions realized by the VCS 30 include the following functions in addition to the functions realized by the VCS 30 according to the first embodiment.

[0142] First, the processor 31 has a function of accepting the selection of a form for inputting a destination. For example, when the form image is corrected, the processor 31 causes one of the input / output devices 40 through the input / output interface 36 to display a screen (selection screen) for accepting the selection of the form.

[0143] Fig. 12 shows an example of a selection screen displayed by the input / output device 40. As shown in Fig. 12, the selection screen displays icons 401 to 403 for selecting a form superimposed on the luggage image 200. In the example shown in Fig. 12, the selection screen displays icons 401 to 403 including numerical values ​​on forms 101 to 103, respectively.

[0144] The operator taps on any of the icons 401 to 403 or inputs a numerical value via the operation unit 41 to select a form. The processor 31 receives an input of an operation to select a form from an operator via the operation unit 41 .

[0145] The processor 31 also has a function of displaying a corrected form image corresponding to a selected form and receiving an input of a destination from an operator.

[0146] When the selection of the form is accepted, the processor 31 causes one of the input / output devices 40 to display a corrected form image corresponding to the selected form through the input / output interface 36. That is, the processor 31 causes the display unit 42 of the input / output device 40 to display the corrected form image corresponding to the selected form.

[0147] When the display unit 42 displays the corrected form image corresponding to the selected form, the operator visually checks the corrected form image corresponding to the selected form and inputs the address written on the form into the operation unit 41 of the input / output device 40.

[0148] The processor 31 acquires the destination input to the operation unit 41 via the input / output interface 36 .

[0149] When the destination is acquired, the processor 31 generates destination information indicating the acquired destination. When the destination information is generated, the processor 31 transmits the generated destination information to the control device 20 through the communication unit .

[0150] Next, an example of the operation of the VCS 30 will be described. FIG. 13 is a flowchart for explaining an example of the operation of the VCS 30.

[0151] First, the processor 31 of the VCS 30 receives the OCR intermediate information and the luggage image from the control device 20 through the communication unit 35 (S41). When the OCR intermediate information and the luggage image are received from the control device 20, the processor 31 determines whether the OCR intermediate information includes form information (S42).

[0152] If it is determined that the OCR intermediate information includes form information (S42, YES), the processor 31 extracts a form image from the package image based on the form information included in the OCR intermediate information (S43). After extracting the form image, the processor 31 corrects the size and inclination of the form image based on the OCR intermediate information (S44).

[0153] After correcting the size and inclination of the form image, the processor 31 accepts input of an operation to select a form through the input / output interface 36 (S45). Upon accepting input of an operation to select a form, the processor 31 causes one of the input / output devices 40 to display the corrected form image corresponding to the selected form (S46).

[0154] When the corrected form image corresponding to the selected form is displayed on one of the input / output devices 40, the processor 31 accepts an input of a destination from the operator via the input / output interface 36 (S47).

[0155] When it is determined that the OCR intermediate information does not include form information (S42, NO), the processor 31 causes one of the input / output devices 40 to display a package image via the input / output interface 36 (S48). After causing one of the input / output devices 40 to display the package image, the processor 31 accepts input of a destination from an operator via the input / output interface 36 (S49).

[0156] When the processor 31 receives input of a destination from the operator (S47 or S49), the processor 31 transmits destination information indicating the destination input in S47 or S49 to the control device 20 via the communication unit 35 (S50). After transmitting the destination information to the control device 20, the processor 31 ends the operation.

[0157] The processor 31 may accept an input of an operation for selecting multiple forms. For example, when the processor 31 accepts an input of a destination written on one form, the processor 31 may again accept an input of an operation for selecting a form. That is, the processor 31 may repeat S45 to S47 multiple times in accordance with the operation of the operator.

[0158] Furthermore, the recognition system 1 may have the features of the recognition system 1 according to the second embodiment.

[0159] The recognition system configured as described above selects a form in accordance with an operation by an operator. The recognition system displays a corrected form image corresponding to the selected form and accepts input of a destination from the operator. As a result, the recognition system can improve the input efficiency of the operator.

[0160] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0161] 1...recognition system, 2...classification device, 3...management device, 4...camera, 10...OCR device, 11...processor, 12...ROM, 13...RAM, 14...NVM, 15...communication unit, 16...operation unit, 17...display unit, 18...camera interface, 20...control device, 21...processor, 22...ROM, 23...RAM, 24...NVM, 25...communication unit, 26...operation unit, 27...display unit, 30...VCS, 31...processor, 32...ROM, 33...RAM, 34...NVM, 35...communication unit, 36...input / output interface, 40...input / output device, 41...operation unit, 42...display unit, 100...luggage, 101...form, 102...form, 103...form, 200...luggage image, 201...form image, 201'...form image, 202...form image, 203...form image, 203'...form image, 301...character line area, 302...character line area, 303...character line area, 401...icon, 402...icon, 403...icon.

Claims

1. An interface for transmitting and receiving data to and from an external device and an input / output device; receiving OCR intermediate information generated in the process of OCR processing for an image including a form on which text lines are written from the external device through the interface; correcting a display method of an image of the document showing the document based on the OCR intermediate information; displaying the corrected form image on the input / output device through the interface; Acquire characters input to the input / output device through the interface; transmitting character information based on the acquired character to the external device through the interface; A processor; An information processing device comprising:

2. The processor corrects the size or inclination of the form image. The information processing device according to claim 1 .

3. the OCR intermediate information includes a size and an inclination of a character line area in which the character line is captured in the image; 3. The information processing device according to claim 1 or 2.

4. The processor corrects the size of the form image based on the height of the character line area. The information processing device according to claim 3 .

5. the processor corrects the inclination of the form image so that one side of the character line area has a predetermined inclination.

5. The information processing device according to claim 3.

6. the OCR intermediate information includes form information relating to a form area in which the form is captured, The processor, receiving the image from the external device through the interface; extracting the form image from the image based on the form information; The information processing device according to claim 1 .

7. The processor receives the form image from the external device through the interface. The information processing device according to claim 1 .

8. The processor, Accepting an input of an operation for selecting the form through the interface; displaying the corrected form image corresponding to the selected form on the input / output device through the interface; The information processing device according to claim 1 .

9. The line of text is a destination. The information processing device according to claim 1 .

10. A program executed by a processor, The processor, A function of receiving OCR intermediate information generated in the process of OCR processing for an image including a form on which text lines are written from an external device; a function of correcting a display method of a form image showing the form based on the OCR intermediate information; A function of displaying the corrected form image on an input / output device; A function of acquiring characters input to the input / output device; a function of transmitting character information based on the acquired character to the external device; A program to achieve this.

Images