Recognition devices, programs, and systems

Abstract

To provide a recognition device, a program, and a system that can effectively perform character recognition processing on a character string image.SOLUTION: According to an embodiment, a recognition device comprises an image interface, an input interface, and a processor. The image interface acquires a character string image including character strings from an input device. The input interface inputs an operation signal to the input device. The processor extracts an area of the character strings from the character string image, acquires the size of the area, inputs a deformation operation to deform the character string image based on the size to the input device through the input interface, acquires the character string image after deformation through the image interface, performs character recognition processing on the character string image after deformation, and inputs the character strings to the input device based on a result of the character recognition processing through the input interface.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to a recognition device, a program, and a system.

Background Art

[0002] There is provided a system that acquires an input screen including a string image including a string such as a destination and an input field for the string from an existing VCD (Video Coding Desk). Such a system extracts a region including a string from the input screen and performs character recognition processing (OCR (Optical Character Recognition) processing) on the extracted region. Based on the result of the OCR processing, the system inputs a key operation for inputting a string into the input field to the existing VCD.

[0003] Conventionally, the system may fail in the OCR processing depending on the position, orientation, size, etc. of the string included in the input screen.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In order to solve the above problems, there is provided a recognition device, a program, and a system that can effectively perform character recognition processing on a string image.

Means for Solving the Problems

[0006] According to one embodiment, the recognition device comprises an image interface, an input interface, and a processor. The image interface acquires a string image containing a string from an input device. The input interface inputs an operation signal to the input device. The processor extracts the area of ​​the string from the string image, obtains the size of the area, inputs a transformation operation to the input device via the input interface to transform the string image based on the size, acquires the transformed string image via the image interface, performs character recognition processing on the transformed string image, and inputs the string to the input device via the input interface based on the result of the character recognition processing. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a block diagram showing an example configuration of a recognition system according to an embodiment. [Figure 2] Figure 2 is a block diagram showing an example configuration of the first recognition device according to this embodiment. [Figure 3] Figure 3 is a block diagram showing an example configuration of an existing VCD according to the embodiment. [Figure 4] Figure 4 is a block diagram showing an example configuration of a second recognition device according to this embodiment. [Figure 5] Figure 5 shows an example of an input screen according to the embodiment. [Figure 6] Figure 6 is a flowchart showing an example of operation of the first recognition device according to this embodiment. [Figure 7] Figure 7 is a flowchart showing an example of the operation of an existing VCD according to the embodiment. [Figure 8] Figure 8 is a flowchart showing an example of operation of the second recognition device according to this embodiment. [Modes for carrying out the invention]

[0008] The embodiments will be described below with reference to the drawings. The recognition system according to this embodiment recognizes a string from a captured image (string image) that contains a string such as a destination.

[0009] The recognition system performs optical character recognition (OCR) processing on the captured image using the first recognition device. If the OCR processing is successful, the recognition system obtains a string of characters based on the results of the OCR processing. If the OCR processing fails, the recognition system displays an input screen including the captured image and input fields on an existing VCD and accepts key input of a string of characters into the input fields.

[0010] While the recognition system normally receives key input of a string from an operator via an existing VCD, in this embodiment, it receives key input from a second recognition device.

[0011] The recognition system uses a second recognition device to acquire the input screen and perform OCR processing on the captured image. If the OCR processing is successful, the recognition system inputs the character strings into the input fields from the second recognition device to the existing VCD based on the results of the OCR processing.

[0012] If the OCR processing fails in the second recognition device, the recognition system displays the acquired input screen on a display unit connected to the second recognition device. The recognition system receives key input from the operator through an operation unit connected to the second recognition device. The recognition system inputs this key input into the existing VCD.

[0013] Figure 1 shows an example of the configuration of the recognition system 1 according to the embodiment. As shown in Figure 1, the recognition system 1 includes a sorter 2, a camera 3, a keyboard / mouse emulator 4 (4a to 4d), a capture board 5 (5a to 5d), a first recognition device 10, an existing VCD 20 (20a to 20d), a second recognition device 30 (30a to 30d), an operation unit 40 (40a and 40b), and a display unit 50 (50a and 50b), etc.

[0014] The first recognition device 10 is connected to the sorter 2, the camera 3, and the existing VCD 20. The existing VCDs 20a to 20d are respectively connected to the keyboard / mouse emulators 4a to 4d, and the capture boards 5a to 5d. The keyboard / mouse emulators 4a to 4d, and the capture boards 5a to 5d are respectively connected to the second recognition devices 30a to 30d. The second recognition devices 30a and 30b are respectively connected to the operation units 40a and 40b. Also, the second recognition devices 30a and 30b are respectively connected to the display units 50a and 50b.

[0015] Note that the recognition system 1 may further include a configuration as required in addition to the configuration shown in FIG. 1, or a specific configuration may be excluded from the recognition system 1.

[0016] Based on the signal from the first recognition device 10, the sorter 2 classifies the incoming articles into sorting destinations. For example, the sorter 2 has a plurality of chutes as sorting destinations. The sorter 2 inputs the articles into the chutes based on the signal from the first recognition device 10. For example, the sorter 2 obtains sorting information indicating the ID for identifying the article and the sorting destination (such as the chute number) for inputting the article from the first recognition device 10. The sorter 2 inputs the articles into the chutes based on the sorting information.

[0017] The camera 3 photographs the articles input into the sorter 2. The camera 3 photographs the surface (destination surface) on which the destination of the article is described as a character string. For example, the camera 3 is installed on the conveying path for inputting the articles into the sorter 2. The camera 3 may photograph the articles from a plurality of surfaces. The camera 3 transmits the photographed image to the first recognition device 10.

[0018] The first recognition device 10 performs OCR processing on an image (captured image) from the camera 3 and recognizes the destination as a character string. The first recognition device 10 sets the sorting destination of the articles for the sorter 2 based on the recognized destination and the like. For example, the first recognition device 10 transmits sorting information indicating the ID for identifying the article and the sorting destination for putting in the article to the sorter 2. The first recognition device 10 will be described in detail later.

[0019] The keyboard / mouse emulator 4 emulates an operation terminal such as a keyboard or a mouse connected to the existing VCD 20. The keyboard / mouse emulator 4 supplies an operation signal similar to the operation signal input by the operator through the operation terminal to the existing VCD 20 according to the control from the second recognition device 30. For example, the keyboard / mouse emulator 4 supplies an operation signal such as mouse movement or click or key input to the existing VCD 20.

[0020] Here, the keyboard / mouse emulators 4a to 4d supply operation signals to the existing VCDs 20a to 20d according to the control of the second recognition devices 30a to 30d, respectively.

[0021] The capture board 5 acquires an input screen from the existing VCD 20. The capture board 5 supplies the acquired input screen to the second recognition device 30.

[0022] Here, the capture boards 5a to 5d acquire the input screens of the existing VCDs 20a to 20d, respectively, and supply them to the second recognition devices 30a to 30d.

[0023] The existing VCD20 is an input device for acquiring destinations included in the captured image (image of the destination surface) when the first recognition device 10 fails to recognize the destination. The existing VCD20 generates an input screen that includes the captured image and an input field. Originally, the existing VCD20 displays the input screen on a monitor and the operator inputs the destination via an operation unit such as a keyboard. Here, the existing VCD20 supplies the input screen to the capture board 5. The existing VCD20 also receives operation signals such as destination key input from the keyboard / mouse emulator 4. The existing VCD20 will be described in detail later.

[0024] The second recognition device 30 acquires the input screen from the capture board 5. The second recognition device 30 recognizes the destination from the captured image contained in the input screen using OCR processing. The second recognition device 30 inputs the recognized destination into the existing VCD 20 via the keyboard / mouse emulator 4. The second recognition device 30 will be described in detail later.

[0025] The control unit 40 receives input for various operations from the operator. The control unit 40 transmits a signal indicating the input operation to the second recognition device 30. The control unit 40 consists of a keyboard, buttons, or a touch panel.

[0026] The display unit 50 displays information based on control from the second recognition device 30. For example, the display unit 50 is composed of a liquid crystal monitor. If the operation unit 40 is composed of a touch panel, the display unit 50 is composed of a liquid crystal monitor integrally formed with the operation unit 40.

[0027] The recognition system 1 may also include an operation unit and a display unit connected to the second recognition devices 30c and 30d, respectively.

[0028] Next, the first recognition device 10 will be described. Figure 2 shows an example configuration of the first recognition device 10. As shown in Figure 2, the first recognition device 10 includes a processor 11, ROM 12, RAM 13, NVM 14, camera interface 15, communication unit 16, operation unit 17, and display unit 18. The processor 11, ROM 12, RAM 13, NVM 14, camera interface 15, communication unit 16, operation unit 17, and display unit 18 are connected to each other via a data bus or a predetermined interface.

[0029] In addition to the configuration shown in Figure 2, the first recognition device 10 may further include any necessary configurations, or certain configurations may be excluded from the first recognition device 10.

[0030] The processor 11 has the function of controlling the operation of the entire first recognition device 10. The processor 11 may also be equipped with an internal cache and various interfaces. The processor 11 performs various processes by executing programs pre-stored in the internal memory, ROM 12, or NVM 14.

[0031] Furthermore, some of the various functions realized by the execution of a program by the processor 11 may be realized by hardware circuits. In this case, the processor 11 controls the functions executed by the hardware circuits.

[0032] ROM12 is a non-volatile memory in which control programs and control data are pre-stored. The control programs and control data stored in ROM12 are pre-programmed according to the specifications of the first recognition device 10.

[0033] RAM13 is volatile memory. RAM13 temporarily stores data being processed by processor 11. RAM13 stores various application programs based on instructions from processor 11. RAM13 may also store data necessary for the execution of application programs and the execution results of application programs.

[0034] NVM14 is a non-volatile memory that allows data to be written to and rewritten. For example, NVM14 is composed of an HDD, SSD, or flash memory. NVM14 stores control programs, applications, and various data according to the operational use of the first recognition device 10.

[0035] The camera interface 15 is an interface for sending and receiving data with the camera 3. For example, the camera interface 15 is connected to the camera 3 by a wire. The camera interface 15 receives images captured by the camera 3. The camera interface 15 transmits the received images to the processor 11. The camera interface 15 may also supply power to the camera 3.

[0036] The communication unit 16 is an interface for sending and receiving data with the sorter 2 and the existing VCD 20, etc. For example, the communication unit 16 supports LAN (Local Area Network) connectivity. Alternatively, for example, the communication unit 16 may support USB (Universal Serial Bus) connectivity. The communication unit 16 may consist of an interface for sending and receiving data with the first recognition device 10 and an interface for sending and receiving data with the existing VCD 20.

[0037] The control unit 17 receives various operation inputs from the operator. The control unit 17 transmits a signal indicating the input operation to the processor 11. The control unit 17 consists of a keyboard, buttons, or a touch panel.

[0038] The display unit 18 displays information based on control from the processor 11. For example, the display unit 18 is composed of a liquid crystal monitor. If the operation unit 17 is composed of a touch panel, the display unit 18 is composed of a liquid crystal monitor integrally formed with the operation unit 17.

[0039] Next, we will explain the existing VCD20. Since the existing VCD20a to 20d have a similar configuration, we will explain them as the existing VCD20.

[0040] Figure 3 shows an example configuration of an existing VCD20. As shown in Figure 3, the existing VCD20 includes a processor 21, ROM 22, RAM 23, NVM 24, communication unit 25, operation interface 26, and display interface 27.

[0041] The processor 21, ROM 22, RAM 23, NVM 24, communication unit 25, operation interface 26, and display interface 27 are connected to each other via a data bus or the like. In addition, the existing VCD20 may have configurations other than those shown in Figure 3 as needed, or certain configurations may be excluded from the existing VCD20.

[0042] The processor 21 has the function of controlling the operation of the entire existing VCD 20. The processor 21 may also be equipped with an internal cache and various interfaces. The processor 21 performs various processes by executing programs pre-stored in the internal memory, ROM 22, or NVM 24.

[0043] Furthermore, some of the various functions realized by the execution of a program by the processor 21 may be realized by hardware circuits. In this case, the processor 21 controls the functions executed by the hardware circuits.

[0044] ROM22 is a non-volatile memory in which control programs and control data are pre-stored. The control programs and control data stored in ROM22 are pre-installed according to the specifications of the existing VCD20.

[0045] RAM23 is volatile memory. RAM23 temporarily stores data being processed by the processor 21. RAM23 stores various application programs based on instructions from the processor 21. RAM23 may also store data necessary for the execution of application programs and the execution results of application programs.

[0046] NVM24 is a non-volatile memory that allows data to be written to and rewritten. For example, NVM24 can be composed of HDDs, SSDs, or flash memory. NVM24 stores control programs, applications, and various data depending on the operational use of the existing VCD20.

[0047] The communication unit 25 (communication interface) is an interface for sending and receiving data with the first recognition device 10 and the like. For example, the communication unit 25 is an interface that supports wired or wireless LAN connections.

[0048] The operation interface 26 is an interface for receiving operation input from an operation terminal. For example, the operation interface 26 receives operation signals indicating operations input to an operation terminal such as a keyboard or mouse. The operation interface 26 supplies the received operation signals to the processor 21. For example, the operation interface 26 supports USB connections.

[0049] Here, the operation interface 26 is connected to the keyboard / mouse emulator 4. That is, the operation interface 26 receives operation signals from the keyboard / mouse emulator 4.

[0050] The display interface 27 is an interface that outputs the screen to a display unit such as a monitor. Here, the display interface 27 is connected to the capture board 5. The display interface 27 is an interface that sends and receives data to and from the capture board 5. The display interface 27 sends the input screen to the capture board 5 according to the control from the processor 21. For example, existing VCD20s include desktop PCs or notebook PCs.

[0051] Next, the second recognition device 30 will be described. The second recognition devices 30a to 30d have a similar configuration and will therefore be described as the second recognition device 30.

[0052] Figure 4 shows an example configuration of the second recognition device 30 according to the embodiment. Figure 4 is a block diagram showing an example configuration of the second recognition device 30. As shown in Figure 4, the second recognition device 30 includes a processor 31, ROM 32, RAM 33, NVM 34, communication unit 35, emulator interface 36, image interface 37, operation interface 38, and display interface 39, etc.

[0053] The processor 31, ROM 32, RAM 33, NVM 34, communication unit 35, emulator interface 36, image interface 37, operation interface 38, and display interface 39 are connected to each other via a data bus or the like. Furthermore, the second recognition device 30 may have additional configurations as needed, in addition to the configuration shown in Figure 4, or certain configurations may be excluded from the second recognition device 30.

[0054] The processor 31 has the function of controlling the operation of the entire second recognition device 30. The processor 31 may also be equipped with an internal cache and various interfaces. The processor 31 performs various processes by executing programs pre-stored in the internal memory, ROM 32, or NVM 34.

[0055] Furthermore, some of the various functions realized by the execution of a program by the processor 31 may be realized by hardware circuits. In this case, the processor 31 controls the functions executed by the hardware circuits.

[0056] ROM32 is a non-volatile memory in which control programs and control data are pre-stored. The control programs and control data stored in ROM32 are pre-programmed according to the specifications of the second recognition device 30.

[0057] RAM33 is volatile memory. RAM33 temporarily stores data being processed by processor 31. RAM33 stores various application programs based on instructions from processor 31. RAM33 may also store data necessary for the execution of application programs and the execution results of application programs.

[0058] NVM34 is a non-volatile memory that allows data to be written to and rewritten. For example, NVM34 can be made up of an HDD, SSD, or flash memory. NVM34 stores control programs, applications, and various data depending on the operational use of the second recognition device 30.

[0059] The communication unit 35 is an interface for sending and receiving data with other second recognition devices 30, etc. For example, the communication unit 35 is an interface that supports wired or wireless LAN connections.

[0060] The emulator interface 36 (input interface) is an interface for sending and receiving data with the keyboard / mouse emulator 4. The emulator interface 36 causes the keyboard / mouse emulator 4 to output operation signals to the existing VCD 20 according to control from the processor 31. In other words, the emulator interface 36 inputs operation signals to the existing VCD 20 through the keyboard / mouse emulator 4. For example, the emulator interface 36 supports USB connection.

[0061] The image interface 37 is an interface for sending and receiving data with the capture board 5. The image interface 37 acquires the input screen of the existing VCD 20 from the capture board 5. The image interface 37 supplies the acquired input screen to the processor 31.

[0062] The operation interface 38 is an interface for sending and receiving data with the operation unit 40. For example, the operation interface 38 receives an operation signal from the operation unit 40 indicating an operation input to the operation unit 40. The operation interface 38 transmits the received operation signal to the processor 31. The operation interface 38 may also supply power to the operation unit 40. For example, the operation interface 38 supports USB connection.

[0063] The display interface 39 is an interface for sending and receiving data with the display unit 50. The display interface 39 outputs image data from the processor 31 to the display unit 50.

[0064] For example, the second recognition device 30 is a desktop PC or a notebook PC. The emulator interface 36, image interface 37, operation interface 38, and display interface 39 (or parts thereof) may be formed as a single unit.

[0065] Furthermore, the second recognition devices 30c and 30d do not necessarily have to be equipped with an operation interface 38 and a display interface 39.

[0066] Next, the functions implemented by the first recognition device 10 will be described. The functions implemented by the first recognition device 10 are achieved by the processor 11 executing a program stored in the ROM 12 or NVM 14, etc.

[0067] First, the processor 11 has the function of acquiring a captured image including the destination surface from the camera 3.

[0068] Here, camera 3 captures an image as the object passes through its shooting area. Camera 3 transmits the captured image to the first recognition device 10.

[0069] The processor 11 acquires the captured image, including the destination surface, from the camera 3 via the camera interface 15. The processor 11 may also send a request to the camera 3 and receive a response that includes the captured image.

[0070] Furthermore, the processor 11 has a function to obtain the destination from the captured image through OCR processing.

[0071] Upon acquiring a captured image, the processor 11 performs OCR processing on the captured image according to a predetermined algorithm (first algorithm). After performing OCR processing, the processor 11 acquires the address printed on the address side of the item based on the results of the OCR processing.

[0072] Furthermore, the processor 11 has a function to obtain the destination via the existing VCD 20 if the OCR processing fails.

[0073] If the OCR process fails and the destination cannot be obtained, the processor 11 transmits the captured image to an existing VCD 20 via the communication unit 16. The processor 11 selects one existing VCD 20 from existing VCDs 20a to 20d and transmits the captured image to the selected existing VCD 20.

[0074] As will be described later, the existing VCD20 inputs the destination written on the destination surface included in the captured image to the first recognition device 10. The processor 11 obtains the destination from the existing VCD 20 via the communication unit 16.

[0075] The processor 11 has a function to set the sorting destination of items based on the destination obtained by OCR processing or the destination entered from an existing VCD 20.

[0076] For example, processor 11 sets the chute number in sorter 2 to which items are loaded as sorting destinations based on the destination. For example, processor 11 sets the chute number corresponding to the administrative division of the destination (such as a prefecture or city / town / village).

[0077] The processor 11 transmits sorting information to the sorter 2 via the communication unit 16, which includes an ID that identifies the item and the sorting destination for that item.

[0078] Next, we will explain the functions that the existing VCD20 provides. The functions that the existing VCD20 provides are achieved by the processor 21 executing a program stored in the ROM 22 or NVM 24, etc.

[0079] First, the processor 21 has the function of acquiring a captured image including the destination surface from the first recognition device 10.

[0080] As mentioned above, if the OCR processing fails, the processor 11 of the first recognition device 10 sends the captured image to the existing VCD 20.

[0081] The processor 21 of the existing VCD20 acquires the captured image from the first recognition device 10 via the communication unit 25.

[0082] Furthermore, the processor 21 has the function of transmitting the input screen, including the acquired captured image, to the capture board 5.

[0083] Upon acquiring a captured image, the processor 21 generates an input screen that accepts input for the destination shown in the captured image. The input screen includes the acquired captured image.

[0084] Figure 5 shows an example of an input screen 100 generated by the processor 21. As shown in Figure 5, the input screen 100 includes an image area 101 and an input field 102, etc.

[0085] Image area 101 displays at least a portion of the captured image acquired from the first recognition device 10. The processor 21 enlarges, reduces, rotates, or crops the captured image and displays it in image area 101. The image resolution of the captured image displayed in image area 101 may be lower than the image resolution of the captured image taken by camera 3.

[0086] In the example shown in Figure 5, image area 101 shows item P. Item P has a form P1 attached to it with the recipient's address written on it. Here, image area 101 displays the side of the form P1 that is attached (the recipient side).

[0087] The input field 102 is located at the bottom of the image area 101. The input field 102 accepts input of a destination that appears in the captured image displayed by the image area 101.

[0088] Furthermore, the input screen 100 may include an icon or the like to confirm the input in the input field 102. Furthermore, the input field 102 may be formed at the top of the image area 101. The configuration of the input screen is not limited to a specific configuration.

[0089] When an input screen is generated, the processor 21 outputs the generated input screen through the display interface 27. The processor 21 outputs the input screen in the same way as when a display device such as a monitor is connected to the display interface 27. That is, the processor 21 outputs a signal to the capture board 5 through the display interface 27 that is the same as the signal output to a display device such as a monitor.

[0090] Furthermore, the processor 21 has a function to input operations (deformation operations) to deform the display of the image area 101 through the operation interface 26.

[0091] For example, a transformation operation is an operation to enlarge, reduce, rotate, or move the captured image displayed in the image area 101.

[0092] For example, the processor 21 receives an operation signal to enlarge the captured image by a predetermined magnification as a deformation operation. Furthermore, the processor 21 receives an operation signal as a deformation operation to reduce the captured image by a predetermined magnification.

[0093] Furthermore, the processor 21 receives an operation signal to rotate the captured image by a predetermined angle as a transformation operation. For example, the processor 21 receives an operation signal to rotate the captured image 90 degrees to the right or left, or to rotate the captured image 180 degrees.

[0094] Furthermore, the processor 21 receives an operation signal to move the captured image horizontally or vertically by a predetermined distance (a predetermined number of pixels) as a transformation operation.

[0095] When a transformation operation is input to the processor 21, the processor 21 updates the input screen according to the input transformation operation. That is, the processor 21 updates the image in the image region 101.

[0096] For example, when the processor 21 receives an operation signal for a transformation operation to enlarge the captured image, it enlarges the captured image beyond the current image in the image area 101 and crops it so that it fits within the image area 101. The processor 21 displays the enlarged and cropped captured image in the image area 101. In addition, the resolution of the captured image within the image area 101 increases due to the enlargement.

[0097] For example, the processor 21 receives a key input (for example, a function key + "1") as a transformation operation through the operation interface 26. The input screen may also display icons related to transformation operations. The processor 21 may detect a tap on an icon as a transformation operation.

[0098] Furthermore, the deformation operations are not limited to a specific configuration. Furthermore, the processor 21 may update the image in the image region 101 by inputting multiple transformation operations.

[0099] Furthermore, the processor 21 has the function of inputting a destination through the operation interface 26.

[0100] When the input screen is displayed, the processor 21 inputs the destination through the operation interface 26. The processor 21 obtains operation signals (operation signals indicating key input, etc.) in the same way as when an operation unit is connected to the operation interface 26. In this case, the processor 21 obtains signals generated by the keyboard / mouse emulator 4 through the operation interface 26.

[0101] Furthermore, the processor 21 has the function of transmitting the input destination to the first recognition device 10.

[0102] When the processor 21 receives an operation signal that confirms input (for example, an operation signal indicating that the Enter key has been pressed) through the operation interface 26, it transmits the input destination to the first recognition device 10 via the communication unit 25.

[0103] Next, the functions implemented by the second recognition device 30 will be described. The functions implemented by the second recognition device 30 are achieved by the processor 31 executing a program stored in the ROM 32 or NVM 34, etc.

[0104] First, the processor 31 has the function of acquiring an input screen from the existing VCD 20 through the image interface 37. Here, the capture board 5 acquires the input screen from the existing VCD 20 and supplies it to the second recognition device 30. The processor 31 acquires the input screen from the capture board 5 via the image interface 37. That is, the processor 31 acquires the captured image, including the destination surface, from the existing VCD 20.

[0105] Furthermore, the processor 31 has a function to acquire the position, orientation, and size of the destination area (destination region) in the captured image included in the input screen.

[0106] Here, the processor 31 obtains the position, orientation, and size of document P1 as the destination area.

[0107] Upon acquiring the input screen, the processor 31 extracts the captured image from the input screen according to the format pre-stored by the NVM 34. That is, the processor 31 extracts the image within the image area 101 of the input screen as the captured image.

[0108] Upon extracting the captured image, the processor 31 obtains the position, orientation, and size of the destination region in the captured image according to a predetermined algorithm.

[0109] For example, the NVM34 pre-stores a model (e.g., a neural network) that takes a captured image as input and outputs the position, orientation, and size of the destination region. The processor 31 inputs the extracted captured image into the model to obtain the position, orientation, and size of the destination region.

[0110] The NVM34 may also pre-store models that output the location of the destination region when a captured image is input, a model that outputs the orientation of the destination region when a captured image is input, and a model that outputs the size of the destination region when a captured image is input. In this case, the processor 31 inputs the extracted captured image to each model to obtain the location, orientation, and size of the destination region.

[0111] Furthermore, the processor 31 has a function to determine whether to input a deformation operation to the existing VCD 20 based on the position, orientation, and size of the destination area.

[0112] In other words, the processor 31 determines whether it can properly perform OCR processing on the captured image in the image region 101.

[0113] For example, the processor 31 determines whether the size of the destination area is greater than or equal to a predetermined size. If the size of the destination area is less than the predetermined size, the processor 31 determines that a transformation operation is input that will make the size of the destination area greater than or equal to the predetermined size. For example, the processor 31 determines that a transformation operation is input that will enlarge the captured image in the image area 101.

[0114] Furthermore, the processor 31 determines whether the orientation of the destination region is correct. That is, the processor 31 determines whether the destination is correct. If it determines that the orientation of the destination region is not correct, the processor 31 determines that it will input a transformation operation to make the orientation of the destination region correct. For example, if the orientation of the destination region is tilted 90 degrees to the left, the processor 31 determines that it will input a transformation operation to rotate the captured image in the image region 101 90 degrees to the right.

[0115] Furthermore, the processor 31 determines whether the destination area is cut off. If it determines that the destination area is cut off, the processor 31 determines to input a transformation operation so that the destination area fits within the image area 101. For example, if the right edge of the destination area is cut off, the processor 31 determines to input a transformation operation to move the captured image within the image area 101 to the left. Also, if the destination area does not fit within the image area 101, the processor 31 determines to input a transformation operation to shrink the captured image within the image area 101.

[0116] Furthermore, the processor 31 may determine when multiple transformation operations are input. For example, the processor 31 may determine when a transformation operation is input that enlarges the captured image within the image region 101 and moves the captured image so that the enlarged destination region fits within the image region 101. The transformation operations that the processor 31 determines to be input are not limited to a specific configuration.

[0117] Furthermore, the processor 31 has the function of inputting an operation signal to the existing VCD 20 that instructs a transformation operation that has been determined to be input.

[0118] When it determines that a transformation operation is to be input, the processor 31 uses the keyboard / mouse emulator 4 to send an operation signal instructing the determined transformation operation to the operation interface 26 of the existing VCD 20. That is, the processor 31 causes the keyboard / mouse emulator 4 to generate an operation signal (for example, a key input) instructing the transformation operation via the emulator interface 36 and output it to the operation interface 26 of the existing VCD 20.

[0119] When a deformation operation signal is input to the existing VCD20, the processor 31 acquires the captured image within the updated (deformed) image region 101 and extracts the destination region.

[0120] Furthermore, the processor 31 has a function to obtain the destination from the destination area through OCR processing.

[0121] Once the destination area (the original destination area or the destination area extracted after updating) is extracted, the processor 31 performs OCR processing on the destination area according to a predetermined algorithm (the second algorithm) different from the first algorithm. The second algorithm can recognize at least some of the characters that the first algorithm cannot recognize.

[0122] When OCR processing is performed, the processor 31 obtains the address written on the address side of the item based on the results of the OCR processing.

[0123] The processor 31 may perform predetermined processing on the image in the destination region before performing OCR processing. For example, the processor 31 may enlarge or reduce the image in the destination region. The processor 31 may also perform processing such as noise removal on the image in the destination region.

[0124] Furthermore, the processor 31 has the function of inputting the destination obtained by OCR processing into the existing VCD 20.

[0125] Once the destination is obtained through OCR processing, the processor 31 uses the keyboard / mouse emulator 4 to send the obtained destination to the operation interface 26 of the existing VCD 20. That is, the processor 31, through the emulator interface 36, causes the keyboard / mouse emulator 4 to generate an operation signal (for example, a key input) to input the destination into the input field 102 and outputs it to the operation interface 26 of the existing VCD 20.

[0126] Alternatively, the processor 31 may input an operation signal to the existing VCD20 indicating the completion of destination input.

[0127] Furthermore, if the OCR processing fails, the processor 31 has a function to input an operation signal indicating the operation input to the operation unit 40 to the operation interface 26.

[0128] If the OCR process fails, the processor 31 displays the input screen from the existing VCD 20 on the display unit 50. Once the input screen is displayed on the display unit 50, the processor 31 accepts input to the operation unit 40. Upon receiving input to the operation unit 40, the processor 31 inputs an operation signal indicating the input operation to the existing VCD 20 via the emulator interface 36.

[0129] Furthermore, the processor 31 may update the input screen on the display unit 50. That is, the processor 31 acquires the input screen from the display interface 27 in real time and displays it on the display unit 50.

[0130] Here, the operator visually inspects the image area of ​​the input screen displayed on the display unit 50 and inputs the destination into the operation unit 40. Once the destination input is complete, the operator inputs an operation to complete the input into the operation unit 40.

[0131] If the operation unit 40 and display unit 50 are not connected to the second recognition device 30, the processor 31 displays the input screen on the display unit 50 connected to the other second recognition device 30. The processor 31 also inputs an operation signal indicating the operation input to the operation unit 40 connected to the other second recognition device 30 to the existing VCD 20.

[0132] For example, the main second recognition device 30 (for example, the second recognition device 30a) or an external control device may manage the operation unit 40 used for inputting the destination and the display unit 50 that displays the input screen.

[0133] Next, an example of the operation of the first recognition device 10 will be described. Figure 6 is a flowchart illustrating an example of the operation of the first recognition device 10.

[0134] First, the processor 11 of the first recognition device 10 acquires a captured image including the target surface of the item through the camera interface 15 (S11). Once the captured image is acquired, the processor 11 performs OCR processing on the captured image according to the first algorithm (S12).

[0135] If the acquisition of the destination by OCR processing fails (S13, NO), the processor 11 transmits the captured image to the existing VCD 20 via the communication unit 16 (S14). After transmitting the captured image to the existing VCD 20, the processor 11 determines via the communication unit 16 whether it has received the destination from the existing VCD 20 (S15).

[0136] If it is determined that no destination has been received from the existing VCD20 (S15, NO), the processor 11 returns to S15.

[0137] If the destination is successfully obtained by OCR processing (S13, YES), or if it is determined that the destination has been received from the existing VCD20 (S15, YES), the processor 11 sets the sorting destination for the items in the sorter 2 based on the destination obtained by OCR processing or the destination received from the existing VCD20 (S16). Once the sorting destination for the items is set in sorter 2, processor 11 terminates its operation.

[0138] Next, we will explain an example of how the existing VCD20 operates. Figure 7 is a flowchart illustrating an example of the operation of the existing VCD20.

[0139] First, the processor 11 of the existing VCD20 determines whether it has received a captured image from the first recognition device 10 via the communication unit 25 (S21). If it determines that it has not received a captured image from the first recognition device 10 (S21, NO), the processor 11 returns to S21.

[0140] When the processor 21 determines that it has received a captured image from the first recognition device 10 (S21, YES), it outputs an input screen including the captured image through the display interface 27 (S22).

[0141] When the input screen is displayed, the processor 21 determines whether a transformation operation has been input through the operation interface 26 (S23). If it is determined that a transformation operation has been input (S23, YES), the processor 21 updates the input screen according to the input transformation operation (S24).

[0142] If it is determined that no transformation operation has been entered (S23, NO), or if the input screen has been updated according to the entered transformation operation (S24), the processor 21 determines whether a destination has been entered through the operation interface 26 (S25). If it is determined that no destination has been entered (S25, NO), the processor 21 returns to S23.

[0143] If the processor determines that a destination has been entered (S25, YES), it transmits the entered destination to the first recognition device 10 via the communication unit 25 (S26). After transmitting the entered destination to the first recognition device 10, the processor 21 terminates its operation.

[0144] Next, an example of the operation of the second recognition device 30 will be described. Figure 8 is a flowchart illustrating an example of the operation of the second recognition device 30.

[0145] The processor 31 of the second recognition device 30 determines whether it has acquired the input screen via the image interface 37 (S31). If it determines that it has not acquired the input screen (S31, NO), the processor 31 returns to S31.

[0146] If it determines that it has obtained the input screen (S31, YES), the processor 31 obtains the position, orientation, and size of the destination area (S32). After obtaining the position, orientation, and size of the destination area, the processor 31 determines whether to input the transformation operation to the existing VCD20 based on the position, orientation, and size of the destination area (S33).

[0147] If it is determined that a deformation operation should be input to the existing VCD20 (S33, YES), the processor 31 inputs the deformation operation signal to the existing VCD20 through the emulator interface 36 (S34).

[0148] When a transformation operation is input to the existing VCD20, the processor 31 obtains the updated input screen through the image interface 37 (S35).

[0149] If it is determined that no deformation operation is to be input to the existing VCD20 (S33, NO), or if the updated input screen is obtained (S35), the processor 31 performs OCR processing on the image of the destination region according to the second algorithm (S36).

[0150] If the destination is successfully obtained through OCR processing (S37, YES), the processor 31 inputs an operation signal to the existing VCD 20 via the emulator interface 36, indicating a key input operation for entering the destination (S38).

[0151] If the acquisition of the destination by OCR processing fails (S37, NO), the processor 31 displays an input screen on the display unit 50 (S39). When the input screen is displayed, the processor 31 inputs an operation signal indicating the operation entered into the operation unit 40 to the existing VCD 20 (S40). Here, the processor 31 continues to execute S40 until it receives input for an operation that has been completed.

[0152] When an operation signal indicating a key input operation for entering a destination is input to the existing VCD20 (S38), or when an operation signal indicating an operation input to the operation unit 40 is input to the existing VCD20 (S40), the processor 31 terminates its operation.

[0153] Furthermore, the processor 31 of the second recognition device 30 does not need to input a deformation operation to the existing VCD 20 to rotate the captured image in the image area 101 by a predetermined angle. In this case, the processor 31 may rotate the image in the destination area to an appropriate orientation before performing OCR processing. Alternatively, the processor 31 may display the input screen with the image in the destination area rotated to an appropriate orientation on the display unit 50.

[0154] Furthermore, if the processor 31 fails to obtain the destination by OCR processing (S37, NO), it may return to S32. In this case, the processor 31 may proceed to S39 if the number of times it has failed to obtain the destination by OCR processing exceeds a predetermined threshold.

[0155] Furthermore, the processor 31 may perform OCR processing on the destination area before proceeding to S32. In this case, if the processor 31 fails to obtain the destination through OCR processing, it may execute S32 to S35.

[0156] The second recognition device 30 may be connected to multiple operating units and multiple display units. Furthermore, the second recognition device 30 may be formed integrally with the operation unit and the display unit.

[0157] Furthermore, the OCR processing by the second algorithm may be performed by an external device. For example, the OCR processing by the second algorithm may be performed by cloud computing. In this case, the processor 31 of the second recognition device 30 transmits the captured image to the external device. The processor 31 obtains the results of the OCR processing from the external device.

[0158] Furthermore, the first recognition device 10 may be formed integrally with the existing VCD 20. Furthermore, the first recognition device 10 may be integrally formed with the camera 3. Furthermore, the first recognition device 10 may be formed integrally with the sorter 2.

[0159] Furthermore, existing VCD20s may also include an operating unit and a display unit. Furthermore, the recognition system 1 may recognize strings other than the address of the item. The strings recognized by the recognition system 1 are not limited to a specific configuration.

[0160] The recognition system configured as described above acquires the position, orientation, and size of the destination region from the input screen displayed by the existing VCD in the second recognition device. Based on the position, orientation, and size of the destination region, the recognition system inputs a transformation operation from the second recognition device to the existing VCD to transform the captured image on the input screen. As a result, the recognition system can acquire a captured image from the input screen in a state suitable for OCR processing in the second recognition device. Therefore, the recognition system can perform OCR processing effectively.

[0161] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. The invention described in the original claims of this application is listed below. [Note 1] An image interface that acquires a string image containing a string from an input device, An input interface for inputting operation signals to the aforementioned input device, Extract the region containing the string from the string image, Obtain the size of the aforementioned region, Through the input interface, a transformation operation to transform the string image based on the size is input to the input device. Through the aforementioned image interface, obtain the transformed string image, Character recognition processing is performed on the transformed string image. The string is input to the input device based on the result of the character recognition process through the input interface. Processor and A recognition device equipped with the following features. [Note 2] If the size is smaller than a predetermined size, the processor inputs a transformation operation to the input device via the input interface to enlarge the character image so that the size of the area becomes the predetermined size. The recognition device described in Appendix 1. [Note 3] The aforementioned processor, The orientation of the aforementioned region is obtained, If the orientation of the region is facing forward, a transformation operation is input to the input device to rotate the string image so that the orientation of the region is facing forward. The recognition device described in Appendix 1 or 2. [Note 4] The aforementioned processor, The position of the aforementioned region is obtained, If the aforementioned region is cut off, a transformation operation is input to the input device to move the text image so that the aforementioned region fits within the text image. The recognition device described in Appendix 1 or 2. [Note 5] The aforementioned string image is an image that failed to undergo character recognition processing in another device. The recognition device described in Appendix 1 or 2. [Note 6] The input interface connects to an emulator that emulates the operating terminal. The recognition device described in Appendix 1 or 2. [Note 7] The above string is the destination, The aforementioned area is the area of ​​the form in which the recipient is written. The recognition device described in Appendix 1 or 2. [Note 8] The image interface acquires an input screen comprising a string image and an input field in which an operator enters the string. The recognition device described in Appendix 1 or 2. [Note 9] An operating interface connected to the control unit, A display interface connected to the display unit, Equipped with, If the character recognition process fails, the processor displays the character string image on the display unit via the display interface and inputs an operation signal indicating the operation input to the operation unit via the input interface to the input device. The recognition device described in Appendix 1 or 2. [Note 10] A program executed by a processor, A function to acquire a string image containing a string from an input device, A function to extract the region of the string from the string image, A function to obtain the size of the aforementioned region, A function to input a transformation operation to the input device that transforms the string image based on the aforementioned size, A function to obtain the transformed string image, The function includes performing character recognition processing on the transformed string image, A function to input the string into the input device based on the result of the character recognition process, A program that makes this possible. [Note 11] A system comprising an input device and a recognition device, The input device is A communication interface for acquiring a string image that failed character recognition processing by the first algorithm, A display interface for outputting images, An operating interface for inputting control signals, The string image is output through the aforementioned display interface, Through the aforementioned operation interface, input a transformation operation to transform the string image, and transform the string image according to the transformation operation. The transformed string image is output through the aforementioned display interface. Equipped with, The aforementioned recognition device is An image interface for acquiring the aforementioned string image from an input device, An input interface for inputting operation signals to the aforementioned input device, Extract the region containing the text from the aforementioned text image, Obtain the size of the aforementioned region, The deformation operation is input to the input device based on the size through the input interface. Through the aforementioned image interface, the transformed string image is obtained, Character recognition processing is performed on the transformed string image according to a second algorithm different from the first algorithm. The string is input to the input device based on the result of the character recognition process through the input interface. Processor and Equipped with, system. [Explanation of symbols]

[0162] 1…Recognition system, 2…Sorter, 3…Camera, 4…Keyboard / mouse emulator, 4a…Keyboard / mouse emulator, 4b…Keyboard / mouse emulator, 4c…Keyboard / mouse emulator, 4d…Keyboard / mouse emulator, 5…Capture board, 5a…Capture board, 5b…Capture board, 5c…Capture board, 5d…Capture board, 10…First recognition device, 11…Processor, 12…ROM, 13…RAM, 14…NVM, 15…Camera interface, 16…Communication unit, 17…Operation unit, 18…Display unit, 20…Existing VCD, 20a…Existing VCD, 20b…Existing VCD, 20c…Existing VCD, 20d…Existing VCD, 21…Pro 22...ROM, 23...RAM, 24...NVM, 25...Communication unit, 26...Operation interface, 27...Display interface, 30...Second recognition device, 30a...Second recognition device, 30b...Second recognition device, 30c...Second recognition device, 30d...Second recognition device, 31...Processor, 32...ROM, 33...RAM, 34...NVM, 35...Communication unit, 36...Emulator interface, 37...Image interface, 38...Operation interface, 39...Display interface, 40...Operation unit, 40a...Operation unit, 40b...Operation unit, 50...Display unit, 50a...Display unit, 50b...Display unit, 100...Input screen, 101...Image area, 102...Input field, NO...Case, P...Item, P1...Form.

Claims

1. An image interface that acquires a string image containing a string from an input device, An input interface for inputting operation signals to the aforementioned input device, Extract the region containing the string from the string image, Obtain the size of the aforementioned region, If the size is smaller than a predetermined size, a transformation operation is input to the input device via the input interface to enlarge the character image so that the size of the area becomes equal to or greater than the predetermined size. Through the aforementioned image interface, obtain the transformed string image, Character recognition processing is performed on the transformed string image. The string is formed based on the result of the character recognition process through the input interface. The above is input to the input device. Processor and A recognition device equipped with the following features.

2. An image interface for acquiring a string image containing a string from an input device, An input interface is connected to an emulator that emulates the operation terminal of the input device, and inputs operation signals to the input device. Extract the region containing the string from the string image, Obtain the size of the aforementioned region, Through the input interface, the string image is deformed based on the size. The deformation operation is input to the input device, Through the aforementioned image interface, obtain the transformed string image, Character recognition processing is performed on the transformed string image. The string is formed based on the result of the character recognition process through the input interface. The above is input to the input device. Processor and A recognition device equipped with the following features.

3. The aforementioned processor, The orientation of the aforementioned region is obtained, If the orientation of the region is facing forward, a transformation operation is input to the input device to rotate the string image so that the orientation of the region is facing forward. The recognition device according to claim 1 or 2.

4. The aforementioned processor, The position of the aforementioned region is obtained, If the aforementioned region is cut off, a transformation operation is input to the input device to move the text image so that the aforementioned region fits within the text image. The recognition device according to claim 1 or 2.

5. The aforementioned string image is an image that failed to undergo character recognition processing in another device. The recognition device according to claim 1 or 2.

6. The above string is the destination, The aforementioned area is the area of ​​the form in which the recipient is written. The recognition device according to claim 1 or 2.

7. The image interface acquires an input screen comprising a string image and an input field in which an operator enters the string. The recognition device according to claim 1 or 2.

8. An operating interface connected to the control unit, A display interface connected to the display unit, Equipped with, If the character recognition process fails, the processor displays the character string image on the display unit via the display interface and inputs an operation signal indicating the operation input to the operation unit via the input interface to the input device. The recognition device according to claim 1 or 2.

9. A program executed by a processor, A function to acquire a string image containing a string from an input device, A function to extract the region of the string from the string image, A function to obtain the size of the aforementioned region, If the aforementioned size is smaller than a predetermined size, a function is provided to input a transformation operation to the input device to enlarge the text image so that the size of the area becomes equal to or greater than the predetermined size. A function to obtain the transformed string image, The function includes performing character recognition processing on the transformed string image, A function to input the string into the input device based on the result of the character recognition process, A program that makes this possible.

10. A program executed by a processor, A function to acquire a string image containing a string from an input device, The input device has a function to connect to an emulator that emulates the operation terminal of the input device and to input operation signals to the input device, A function to extract the region of the string from the string image, A function to obtain the size of the aforementioned region, A function to input a transformation operation to the input device that transforms the string image based on the aforementioned size, A function to obtain the transformed string image, The function includes performing character recognition processing on the transformed string image, A function to input the string into the input device based on the result of the character recognition process, A program that makes this possible.

11. A system comprising an input device and a recognition device, The input device is A communication interface for acquiring a string image that failed character recognition processing by the first algorithm, A display interface for outputting images, An operating interface for inputting control signals, The string image is output through the aforementioned display interface, Through the aforementioned operation interface, input a transformation operation to enlarge the string image. The string image is transformed according to the transformation operation described above. The transformed string image is output through the aforementioned display interface. The first processor, Equipped with, The aforementioned recognition device is An image interface for acquiring the aforementioned string image from an input device, An input interface for inputting operation signals to the aforementioned input device, Extract the region containing the text from the aforementioned text image, Obtain the size of the aforementioned region, If the size is smaller than a predetermined size, the transformation operation to enlarge the character image so that the size of the area is equal to or greater than the predetermined size is input to the input device through the input interface. Through the aforementioned image interface, the transformed string image is obtained, Character recognition processing is performed on the transformed string image according to a second algorithm different from the first algorithm. The string is input to the input device based on the result of the character recognition process through the input interface. The second processor, Equipped with, system.

12. A system comprising an input device and a recognition device, The input device is A communication interface for acquiring a string image that failed character recognition processing by the first algorithm, A display interface for outputting images, An operating interface for inputting control signals, The string image is output through the aforementioned display interface, Through the aforementioned operation interface, input a transformation operation to transform the string image. The string image is transformed according to the transformation operation described above. The transformed string image is output through the aforementioned display interface. The first processor, Equipped with, The aforementioned recognition device is An image interface for acquiring the aforementioned string image from an input device, An input interface is connected to an emulator that emulates the operation terminal of the input device, and inputs operation signals to the input device. Extract the region containing the text from the aforementioned text image, Obtain the size of the aforementioned region, The deformation operation is input to the input device based on the size through the input interface. Through the aforementioned image interface, the transformed string image is obtained, Character recognition processing is performed on the transformed string image according to a second algorithm different from the first algorithm. The string is input to the input device based on the result of the character recognition process through the input interface. The second processor, Equipped with, system.

Images