Coding device and coding processing program

Abstract

To provide a coding device and a coding processing program capable of improving a recognition rate of a character string.SOLUTION: According to an embodiment, a coding device has a communication unit and a processor. The communication unit acquires an original image selected from a plurality of captured images obtained by photographing an article and thumbnail images of the plurality of captured images. If the processor fails to recognize a character string of a destination for the original image, the processor detects a destination area from the thumbnail images of the captured images other than the original image, replaces the captured image of the thumbnail image in which the destination area has been detected with the original image, and performs processing of recognizing the character string of the destination.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] An embodiment of the present invention relates to a coding device and a coding process program. [Background technology]

[0002] Conventionally, there is a coding device called a video coding disk (VCD) for identifying the destination of an image in which the destination character string cannot be recognized by an article processing system. Recently, there is an article processing system that performs character recognition by photographing an article to be processed from multiple directions and detecting a photographed image (original image) containing the destination character string from the multiple photographed images. When the destination character string cannot be recognized, such an article processing system supplies the photographed image other than the original image as a thumbnail image to the VCD together with the original image that was the subject of the recognition of the destination character string.

[0003] A conventional VCD recognizes the destination character string by performing character recognition on the original image from the article processing system using an algorithm different from that of the article processing system, and if the recognition fails, the operator is made to input the destination. However, it is possible that the original image detected by the article processing system does not have a destination, and the captured image used as a thumbnail image has the destination written therein. With a conventional VCD, when an operator finds a thumbnail image with a destination character string, the operator can specify the thumbnail image with the destination. In this case, the VCD is configured to accept the destination input by the operator in a state where the captured image of the specified thumbnail image is replaced with the original image and displayed. In other words, a conventional VCD has a problem in that the operator's workload increases when the original image does not have a destination and the thumbnail image has a destination.

[0004] In recent years, a system has also been proposed in which a second coding device is connected to an existing VCD and the destination character string is specified by character recognition in the second coding device or by key input by an operator. The second coding device captures an image displayed on the display unit of the existing VCD and performs character recognition on the captured image, which causes a problem that the operator's workload increases when the destination is not in the original image but is in the thumbnail image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-140632 A Summary of the Invention [Problem to be solved by the invention]

[0006] In order to solve the above problems, a coding device and a coding processing program are provided that can improve the recognition rate of a character string. [Means for solving the problem]

[0007] According to an embodiment, a coding device includes a communication unit and a processor. The communication unit acquires an original image selected from a plurality of images of an article and thumbnail images of the plurality of images. If the processor fails to recognize a character string of a destination in the original image, the processor detects a destination area from thumbnail images of the images other than the original image, replaces the photographed image of the thumbnail image in which the destination area was detected with the original image, and performs a process of recognizing the character string of the destination. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a recognition system including a coding device according to a first embodiment. [Diagram 2]FIG. 2 is a block diagram showing an example of the configuration of a control device of an article processing system included in the recognition system according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram showing an example of the configuration of a VCD as a coding device according to the first embodiment. [Figure 4] FIG. 4 is a flowchart for explaining a first operation example by a VCD as a coding device according to the first embodiment. [Diagram 5] FIG. 5 is a diagram showing a first display example of a display screen displayed on a display unit by a VCD as a coding device according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of the relationship between thumbnail images and a plurality of captured images displayed on a display screen by the coding device according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing a second display example of a display screen displayed on the display unit by a VCD as the coding device according to the first embodiment. [Figure 8] FIG. 8 is a flowchart for explaining a second operation example by the VCD as the coding device according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of the configuration of a recognition system including a coding device according to the second embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a second coding device as a coding device according to the second embodiment. [Figure 11] FIG. 11 is a flowchart for explaining a first operation example of the second coding device as the coding device according to the second embodiment. [Figure 12] FIG. 12 is a flowchart for explaining a second operation example of the second coding device as the coding device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Each embodiment will be described below with reference to the drawings. (First embodiment) First, the first embodiment will be described. FIG. 1 is a diagram showing an example of the configuration of a recognition system 101 including a coding device according to a first embodiment. The recognition system 101 according to the first embodiment has an article processing system 1 and a VCS (video coding system) 2. The article processing system 1 is a system that processes an article based on destination information displayed somewhere on a three-dimensional article. The VCS 2 is a system that identifies destination information included in an image from the article processing system 1 and returns the identified destination information to the article processing system 1.

[0010] In the configuration example shown in FIG. 1, the article processing system 1 has a sorter 6, a camera 7, and a control device 8. The control device 8 is connected to the sorter 6 and the camera 7, and is communicably connected to the VCS 2. The control device 8 has a function of performing character recognition processing to recognize character strings that become destination information. The control device 8 uses a recognition device to recognize characters as destination information in character recognition for images captured by the camera 7. The sorter 6 is an example of a processing device that processes items. The sorter 6 performs a process of dividing items to be processed into sorting destinations based on a control signal from the control device 8. For example, the sorter 6 sorts items supplied to a predetermined position into sorting destinations according to destination information written on the items based on a control signal from the control device 8. The sorter 6 sorts each item into a sorting destination specified by the control device 8. For example, the sorter 6 obtains sorting information from the control device 8 that indicates an ID that identifies an item and information that indicates the sorting destination of the item (e.g., a chute number, etc.). The sorter 6 performs a sorting process on the items based on the sorting information obtained from the control device 8.

[0011] Camera 7 photographs the item to be processed by sorter 6. In this embodiment, the item to be processed is assumed to be three-dimensional, and a character string of destination information is written (displayed) on one of the faces of the three-dimensional object. Camera 7 photographs the three-dimensional object from multiple directions. Camera 7 is installed to obtain multiple images of the item to be processed by sorter 6 taken from multiple directions. Camera 7 may be configured to obtain images taken from multiple directions using multiple cameras, or one camera may be moved to obtain multiple images taken from multiple directions. Camera 7 transmits the multiple images it has taken to control device 8.

[0012] In this embodiment, the article to be processed is assumed to be an article on which destination information is written on a slip in a specified format and attached to the surface of the article. However, the article to be processed may be any article on which characters as destination information are displayed somewhere on the surface, and may be an article on which a slip in any format on which destination information is written is attached, or an article on which a destination display is written or printed on the surface of the article.

[0013] The control device 8 controls the article processing system 1. The control device 8 acquires a plurality of captured images of the article to be processed captured from a plurality of directions by the camera 7. The control device 8 detects an image including a character string of destination information from the plurality of captured images as an original image. The control device 8 performs character recognition processing to recognize the character string of destination information from the original image detected from the plurality of captured images.

[0014] If the control device 8 fails to recognize the destination information, it transmits an original image of the item and a thumbnail image of a captured image other than the original image to the VCS 2, and acquires the destination information of the item from the VCS 2. The control device 8 sets a sorting destination for the item in the sorter 6 based on the recognized destination information or the destination information acquired from the VCS 2. For example, the control device 8 transmits sorting information including an ID for identifying the item and information indicating the sorting destination of the item to the sorter 6. In this way, the control device 8 controls the processing of the item based on the destination information.

[0015] In addition, in the configuration example shown in Fig. 1, the VCS 2 has a plurality of VCDs 20 (20a, 20b, 20c, 20d). Each VCD 20 (20a, 20b, 20c, 20d) is connected to an operation unit 30 (30a, 30b, 30c, 30d) and a display unit 40 (40a, 40b, 40c, 40d). The operation unit 30 is an operation device that operates operation keys by which an operator inputs destination information, etc. The display unit 40 is a display device that displays an image of an article for which destination information is to be input by the operation unit 30.

[0016] Each VCD 20 performs character recognition processing (OCR processing) on ​​the captured image supplied from the article processing system 1. In this embodiment, each VCD 20 obtains an original image, which is one of multiple captured images of an article, from the control device 8 of the article processing system 1, and a thumbnail image of a captured image other than the original image. The original image is an image that the control device 8 has determined to include a character string of a destination (an image that the control device 8 has targeted for character recognition), and the thumbnail image is a captured image other than the original image.

[0017] The VCD 20 performs character recognition (second character recognition) on the original image acquired from the control device 8 using an algorithm different from that of the control device 8. If the VCD 20 is able to recognize a character string of a destination in the second character recognition on the original image, it transmits destination information, which is the recognition result of the second character recognition, to the control device 8.

[0018] Furthermore, if the VCD20 fails to recognize the destination character string by the second character recognition, it performs a process of detecting the area of ​​the destination character string (destination area) in the thumbnail image. If the VCD20 can detect the destination area from the thumbnail image, it replaces the thumbnail image in which the destination area was detected with an original image, and performs character recognition on the image in which the destination area was detected as the original image. If the VCD20 can recognize the destination character string by character recognition of the replaced original image, it transmits destination information, which is the recognition result of the character recognition, to the control device 8.

[0019] If the VCD 20 cannot recognize the destination character string even after replacing the original image, or if the destination area cannot be detected from the thumbnail image, the VCD 20 displays the image acquired from the control device 8 on the display unit 40 and allows the operator to key in the destination information using the operation unit 30. When the operator keys in the destination information using the operation unit 30, the VCD 20 transmits the destination information input by the operator to the control device 8.

[0020] Next, the configuration of a control system in the control device 8 of the article processing system 1 in the recognition system 101 according to the first embodiment will be described. FIG. 2 is a block diagram showing an example of the configuration of a control system in the control device 8 of the article processing system 1 in the recognition system 101 according to the first embodiment. As shown in FIG. 2, the control device 8 includes a processor 11, a ROM 12, a RAM 13, an NVM 14, a camera interface (I / F) 15, a communication unit 16, an operation unit 17, a display unit 18, a sorter interface (I / F) 19, and the like.

[0021] The processor 11 is communicatively connected to the ROM 12, RAM 13, NVM 14, camera interface 15, communication unit 16, operation unit 17, display unit 18, and sorter interface 19 via a data bus, an interface, etc. The control device 8 may further include components as necessary in addition to the components shown in FIG. 2, or certain components may be excluded from the control device 8.

[0022] The processor 11 has a function of controlling the overall operation of the control device 8. The processor 11 may include an internal cache and various interfaces. The processor 11 realizes various processes by executing programs stored in advance in the internal memory, the ROM 12, or the NVM 14. Some of the various functions realized by the processor 11 executing the programs may be realized by hardware circuits. In this case, the processor 11 controls the functions executed by the hardware circuits.

[0023] The ROM 12 is a non-volatile memory in which control programs, control data, etc. are stored in advance. The control programs and control data stored in the ROM 12 are incorporated in advance in accordance with the specifications of the control device 8.

[0024] The RAM 13 is a volatile memory. The RAM 13 temporarily stores data being processed by the processor 11. The RAM 13 stores various application programs based on instructions from the processor 11. The RAM 13 may also store data necessary for executing the application programs and execution results of the application programs.

[0025] The NVM 14 is a non-volatile memory in which data can be written and rewritten. For example, the NVM 14 is configured with a HDD, an SSD, a flash memory, etc. The NVM 14 stores control programs, applications, various data, etc. according to the operational purpose of the control device 8.

[0026] The camera interface 15 is an interface for transmitting and receiving data to and from the camera 7. The camera interface 15 is an interface for connecting and communicating with the camera 7 via wired or wireless communication. The camera interface 15 receives images captured by the camera 7 from the camera 7. The camera interface 15 transmits the received captured images to the processor 11. The camera interface 15 may also supply power to the camera 7.

[0027] In this embodiment, the camera 7 photographs the items to be processed by the sorter 6 from multiple directions. Therefore, the camera 7 may be composed of multiple camera devices (imaging units) that photograph the items at a predetermined position from multiple directions. When the camera 7 is multiple camera devices, the camera interface 15 includes an interface for communication connection to the multiple camera devices as the camera 7.

[0028] The communication unit 16 is an interface for transmitting and receiving data to and from each VCD 20 in the VCS 2. For example, the communication unit 16 supports a LAN (Local Area Network) connection. The communication unit 16 may also support a USB (Universal Serial Bus) connection. The communication unit 16 may be an interface for connecting to a communication control device such as a router or switching hub for connecting to a plurality of VCDs 20, as long as it has an interface for transmitting and receiving data to and from the control device 8.

[0029] The operation unit 17 receives various operation inputs from an operator. The operation unit 17 transmits a signal indicating the input operation to the processor 11. The operation unit 17 is composed of a keyboard, buttons, a touch panel, or the like.

[0030] The display unit 18 displays information under the control of the processor 11. For example, the display unit 18 is configured with a liquid crystal monitor. When the operation unit 17 is configured with a touch panel, the display unit 18 is configured with a liquid crystal monitor formed integrally with the operation unit 17.

[0031] The sorter interface 19 is an interface for transmitting and receiving data to and from the sorter 6. For example, the sorter interface 19 is communicatively connected to the sorter 6 via wired or wireless communication. The sorter interface 19 supplies the sorter 6 with sorting information based on destination information for items to be processed by the sorter 6. The sorter interface 19 may also receive information indicating the processing status in the sorter 6 from the sorter 6. The sorter interface 19 may also be configured to be included in the communication unit 16.

[0032] Next, the configuration of the VCD 20 in the VCD 2 serving as the coding device according to the first embodiment will be described. 3 is a block diagram showing an example of the configuration of a control system in a VCD 20 as a coding device according to the first embodiment. Since VCDs 20a to 20d can be realized with the same configuration, they will be described below as VCD 20.

[0033] 3, the VCD 20 includes a processor 21, a ROM 22, a RAM 23, an NVM 24, a communication unit 25, an operation interface (I / F) 26, and a display interface (I / F) 27. The processor 21 is connected to the ROM 22, the RAM 23, the NVM 24, the communication unit 25, the operation interface 26, and the display interface 27 via a data bus or the like. The VCD 20 may include a configuration as necessary in addition to the configuration shown in FIG. 3, or a specific configuration may be excluded from the configuration shown in FIG. 3.

[0034] The processor 21 controls the overall operation of the VCD 20. The processor 21 may also include an internal cache and various interfaces in addition to an operator that executes a program. The processor 21 realizes various processes by executing programs stored in the internal memory, the ROM 22, or the NVM 24. Some of the various functions realized by the processor 21 executing the programs may be realized by a hardware circuit. In this case, the processor 21 controls the functions executed by the hardware circuit.

[0035] The ROM 22 is a non-volatile memory that stores control programs, control data, etc. The control programs and control data stored in the ROM 22 are incorporated in accordance with the specifications of the VCD 20.

[0036] The RAM 23 is a volatile memory. The RAM 23 temporarily stores data being processed by the processor 21. The RAM 23 stores various application programs based on instructions from the processor 21. The RAM 23 may also store data necessary for executing the application programs and execution results of the application programs. The RAM 23 may also be used as a buffer memory that temporarily stores data to be transmitted or received.

[0037] The NVM 24 is a non-volatile memory to which data can be written and rewritten. For example, the NVM 24 is composed of a HDD, an SSD, a flash memory, etc. The NVM 24 stores control programs, applications, various data, etc. according to the operational use of the VCD 20.

[0038] The communication unit 25 is a communication interface for transmitting and receiving data to and from the control device 8, etc. For example, the communication unit 25 supports a LAN (Local Area Network) connection by wired or wireless communication. The communication unit 25 may also support a USB (Universal Serial Bus) connection.

[0039] The operation interface 26 is an interface for connecting to the operation unit 30. For example, the operation interface 26 receives an operation signal indicating an operation input to the operation unit 30 which is configured with a keyboard or a mouse operated by an operator. In addition, the operation interface 26 supplies the operation signal received from the operation unit 30 to the processor 21.

[0040] The display interface 27 is an interface for connecting to a display unit 40 that displays a display screen to be viewed by an operator. The display interface 27 is an interface for outputting image data indicating a display screen to be displayed on the display unit 40 under the control of the processor 21.

[0041] Next, the operation of the control device 8 of the article processing system 1 in the recognition system 101 according to the first embodiment will be described. The control device 8 performs various processes for controlling the operation of the item processing system 1 by the processor 11 executing a program stored in the ROM 12 or NVM 14, etc. In this embodiment, the processor 11 of the control device 8 has a character recognition function for recognizing a character string of a destination in an image captured by the camera 7.

[0042] When the article processing system 1 processes one article, the processor 11 of the control device 8 acquires a plurality of captured images of the article to be processed from a plurality of directions by the camera 7. Here, the camera 7 captures images of the article located at a predetermined photographing position (photographing area) from a plurality of directions.

[0043] For example, the camera 7 is composed of multiple camera devices installed at multiple positions so that an article in a predetermined shooting position can be shot from multiple directions. The camera 7 may also be composed of one or multiple camera devices that shoot an article from multiple directions while moving a position relative to the article in a predetermined shooting area. The processor 11 of the control device 8 according to the first embodiment acquires multiple captured images of one article to be processed by the camera 7 from multiple directions via the camera interface 15.

[0044] When processor 11 acquires a plurality of captured images taken by camera 7, processor 11 performs a recognition process (first recognition process) that recognizes the destination from the captured images by character recognition processing using a predetermined algorithm (first algorithm). In the first authentication process, processor 11 performs a destination area detection process that detects a captured image that includes an area of ​​the destination character string from the plurality of captured images. Processor 11 selects one captured image in which the destination area is detected as an original image, and performs a process of recognizing the destination character string in the original image. If processor 11 is successful in recognizing the destination character string through the first recognition process, processor 11 identifies the character string obtained as a result of character recognition as the destination information of the item.

[0045] Furthermore, if the first recognition process fails to recognize the character string of the destination, the processor 11 performs a process of identifying the destination information using the VCS 2. That is, if the first recognition process fails to recognize the character string of the destination, the processor 11 transmits coding information including the original image and thumbnail images of each captured image to the VCS 2 via the communication unit 16. The coding information is information that requests the VCD 20 in the VCS 2 to identify (code) the destination information, and includes an image for coding process for recognizing or keying in the destination of the item to be processed.

[0046] The image for coding processing includes one photographed image (original image) selected from the multiple photographed images of the article to be processed and thumbnail images of each photographed image. The original image is full-size (e.g., photographed size) image data used for character recognition processing, and the thumbnail image is image data smaller than the full size for reduced display. However, the control device 8 may transmit full-size image data of the multiple photographed images and information indicating original images of the multiple photographed images to the VCD 20 as images for coding processing. In this case, the thumbnail images of the multiple photographed images may be generated by the processor 21 of the VCD 20.

[0047] When the processor 11 of the control device 8 cannot recognize the destination information from the captured image of the article, it selects one VCD 20 from the VCDs 20a to 20d and transmits coding information including an image for coding processing to the selected VCD 20. In response to this, the VCD 20 identifies the destination information from the image for coding processing included in the coding information, and returns the identified destination information to the control device 8. As a result, the processor 11 of the control device 8 obtains the destination information identified by the VCD 20 via the communication unit 16.

[0048] Processor 11 sets the sorting destination of the item based on the destination information identified by the first recognition process or the destination information acquired from VCD 20. For example, processor 11 sets a sorting location in sorter 6 as the sorting destination based on the destination information. For example, processor 11 selects a chute corresponding to the administrative district (such as a prefecture or a city, town, or village) of the destination information of the item to be processed from a plurality of chutes (sorting locations) in sorter 6 into which items are input, and sets information indicating that chute (such as the chute number) as the sorting destination. Processor 11 transmits sorting information including an ID that identifies the item and information indicating the sorting destination of the item identified by the ID to sorter 6 via sorter interface 19. As a result, sorter 6 performs a sorting process of sorting items based on the sorting information supplied from control device 8.

[0049] Next, the operation of the VCD 20 in the VCS 2 of the recognition system 101 according to the first embodiment will be described. The VCD 20 performs coding processing of destination information for multiple captured images from the control device 8 of the item processing system 1 by the processor 21 executing a program stored in the ROM 22 or NVM 24, etc. As described above, if the first recognition processing fails, the processor 11 of the control device 8 transmits coding information including an image for coding processing of the item to the VCD 20.

[0050] FIG. 4 is a flowchart for explaining a first operation example of the VDD 20 as the coding device according to the first embodiment. The VCD 20 receives coding information including multiple images of one item photographed from multiple directions from the control device 8 via the communication unit 25 (step S11). The processor 21 stores the coding information received from the control device 8 via the communication unit 25 in a memory such as the RAM 23 or NVM 24. The coding information includes images for coding processing associated with identification information (ID) of the item. The images for coding processing include an original image, which is full-size image data, and thumbnail images of the multiple photographed images.

[0051] Processor 21 executes a second recognition process (step S12) for recognizing a destination character string by character recognition of a second algorithm different from the character recognition in control device 8 for the original image extracted from the coding information. If processor 21 succeeds in recognizing the destination character string by the second recognition process (step S13, YES), processor 21 transmits destination information generated based on the recognized destination character string to control device 8 (step S14), and ends the process for the coding information (coding process).

[0052] In parallel with the second recognition process for the original image, the processor 21 executes thumbnail processing for recognizing destination information from captured images other than the original image included in the coding information (steps S21-S24). Here, it is assumed that the VCD 20 receives from the control device 8 an original image (full-size image data) designated by the control device 8 as an image for coding process included in the coding information, and thumbnail images (thumbnail-size image data) of a plurality of captured images.

[0053] As thumbnail processing, the processor 21 performs detection processing to detect an area of ​​a character string of a destination (destination area) in thumbnail images of photographed images other than the original image (step S21). For example, if it is assumed that the destination is written on a form of a predetermined format, the processor 21 detects an area of ​​an image resembling a form of the predetermined format in the thumbnail images of each photographed image as the destination area. Also, if it is assumed that the destination is written on the surface of an item in any format, the processor 21 may detect an area of ​​an image of a character group (character string group) resembling a destination as the destination area.

[0054] If the processor 21 cannot detect the destination area from any of the thumbnail images of the captured images (step S22, NO), the processor 21 retains as a processing result of the thumbnail processing a message indicating that the destination area could not be detected from any image other than the original image designated by the control device 8.

[0055] Furthermore, if the processor 21 can detect a destination area from a captured image other than the original image (step S22, YES), it performs a replacement process to replace the captured image in which the destination area was detected with the original image (step S23). For example, when the control device 8 designates the first captured image as the original image, if a destination area is detected from the second captured image of the thumbnail image, the processor 21 obtains full-size image data of the second captured image from the control device 8 and replaces the full-size second captured image with the original image.

[0056] The VCD 20 may receive full-size image data of a plurality of captured images as images for coding processing included in the coding information from the control device 8, and store the image data in a memory such as the RAM 23. In this case, the processor 21 may detect a destination area from a captured image other than the original image, and if the destination area is detected from a captured image other than the original image, may set the full-size image of the second captured image from which the destination information was detected as the original image.

[0057] When the original image is replaced, the processor 21 recognizes the destination character string on the replaced original image by character recognition processing of a second algorithm different from that of the control device 8 (step S24). The processor 21 holds the recognition result of the destination character string on the replaced original image by character recognition processing using the second algorithm as the processing result of the thumbnail processing.

[0058] For example, if the processor 21 cannot recognize the destination character string from the replaced original image, the processor 21 retains as the processing result of the thumbnail processing a message indicating that the destination information could not be recognized from any image other than the original image specified by the control device 8. In addition, if the processor 21 can recognize the destination character string from the replaced original image, the processor 21 retains the destination information including the recognized destination character string as the processing result of the thumbnail processing.

[0059] On the other hand, if processor 21 fails to recognize the destination character string for the original image designated by control device 8 (step S13, NO), processor 21 determines whether or not the thumbnail processing as described above has ended (step S15). If the thumbnail processing has not ended (step S15, NO), processor 21 waits for the thumbnail processing to end.

[0060] If the first recognition process fails and the thumbnail process ends (step S15, YES), the processor 21 determines whether or not the recognition of the destination information is successful as a result of the thumbnail process (step S16). If the recognition of the destination information by the thumbnail process is successful (step S16, YES), the processor 21 transmits the destination information recognized by the thumbnail process to the control device 8 (step S14), and ends the coding process for the coding information.

[0061] Furthermore, if the recognition of the destination information by the thumbnail process fails (step S16, NO), the processor 21 displays an image for coding process of the coding information on the display unit 40, and performs a keystroke input process to have the operator key in the destination information using the operation unit 30 (step S17). Here, the processor 21 displays the original image as a main image, and displays the thumbnail image as a sub-image on the display screen of the display unit 40.

[0062] Fig. 5 is a first display example in which the VCD 20 displays a display screen including an original image and a thumbnail image on the display unit 40. Fig. 6 is a diagram for explaining an example of the arrangement of a plurality of captured images displayed as thumbnail images in a predetermined sub-display area 42 on the display screen. Fig. 7 is a second display example of a display screen displayed by the VCD 20 on the display unit 40 when the original image on the display screen shown in Fig. 5 is replaced.

[0063] In the display examples shown in FIGS. 5 and 7, the display unit 40 displays a display screen (coding screen) including a main display area 41, a sub-display area 42, and an input display area 43. The main display area 41 displays one photographed image to be subjected to character recognition processing selected from multiple photographed images of the article to be processed as an original image. The sub-display area 42 has multiple display areas 421-425, and displays thumbnail images (reduced images) of the multiple photographed images in each of the display areas 421-425. The input display area 43 displays information input by the operator, etc.

[0064] Here, the control device 8 acquires five captured images of a three-dimensional object to be processed captured by the camera 7 from five directions, as exemplified in Fig. 6. In this case, the control device 8 supplies an original image, which is one captured image selected from the five captured images, and thumbnail images of the five captured images to the VCD 20 as images for coding processing. The VCD 20 displays the original image from the control device 8 in the main display area 41, and displays the thumbnail images of the five captured images in each display area of ​​the sub-display area 42.

[0065] In the display examples shown in Figs. 5 to 7, the sub-display area 42 has five display areas 421, 422, 423, 424, and 425 corresponding to five captured images taken from five directions. As shown in Fig. 6, the first display area 421 displays a thumbnail image of a captured image of a first surface (surface captured from a first direction) M1 of an article. The second display area 422 displays a thumbnail image of a captured image of a second surface (surface captured from a second direction) M2 of an article. The third display area 423 displays a thumbnail image of a captured image of a third surface (surface captured from a third direction) M3 of an article. The fourth display area 424 displays a thumbnail image of a captured image of a fourth surface (surface captured from a fourth direction) M4 of an article. The fifth display area 425 displays a thumbnail image of a captured image of a fifth surface (surface captured from a fifth direction) M5 of an article.

[0066] The original image 411 displayed in the main display area 41 is, for example, an original image designated by the control device 8. In addition, the original image 411 displayed in the main display area 41 may be an original image (a photographed image in which a destination area has been detected) replaced by thumbnail processing.

[0067] The original image displayed in the main display area 41 is switched in response to an instruction from the operator via the operation unit 30. For example, in the first display example shown in Fig. 5, the fifth display area 425 is in a selected state, and a full-size image of the captured image, whose thumbnail image is displayed in the fifth display area 425, is displayed in the main display area 41 as the original image 411. In the display state shown in Fig. 5, when the operator instructs to select the thumbnail image displayed in the first display area 421, the processor 21 selects the first display area 421, as in the second display example shown in Fig. 7, and displays the full-size image of the captured image, whose thumbnail image is displayed in the first display area 421, in the main display area 41 as the original image 412.

[0068] The operator keys in the destination information using the operation unit 30 while visually checking (referring to) the display screen shown in Fig. 5 or 7. The processor 21 of the VCD 20 obtains the destination information keyed in by the operator using the operation unit 30. When the operator keys in the destination information using the operation unit 30, the processor 21 transmits the destination information input using the operation unit 30 to the control device 8 (step S14), and ends the coding process for the coding information.

[0069] Next, a second operation example of the VDD 20 as the coding device according to the first embodiment will be described. In the second operation example, the thumbnail processing executed as parallel processing in the first operation example is executed when recognition of the destination character string for the original image designated by the control device 8 fails. FIG. 8 is a flowchart for explaining a second operation example of the VDD 20 as the coding device according to the first embodiment.

[0070] The VCD 20 receives coding information including a plurality of images of an article photographed from a plurality of directions from the control device 8 via the communication unit 25 (step S31). The processor 21 stores the coding information including the original image and thumbnail images of the plurality of photographed images received from the control device 8 via the communication unit 25 in a memory such as the RAM 23 or the NVM 24.

[0071] Processor 21 performs a second recognition process (step S32) for recognizing a destination character string by character recognition of a second algorithm different from the character recognition by control device 8 for an original image extracted from the coding information. If processor 21 succeeds in recognizing the destination character string by the second recognition process (step S33, YES), processor 21 transmits destination information generated based on the recognized destination character string to control device 8 (step S34), and ends the process for the coding information (coding process).

[0072] If processor 21 fails to recognize the destination character string for the original image designated by control device 8 (step S33, NO), processor 21 executes thumbnail processing as described in the first operation example (steps S41-S44). As thumbnail processing, processor 21 first performs destination area detection processing for thumbnail images of captured images other than the original image (step S41). If processor 21 cannot detect the destination area from the thumbnail image (step S42, NO), processor 21 determines the thumbnail processing result to the effect that the destination area could not be detected from images other than the original image, and proceeds to step S46.

[0073] If the processor 21 can detect a destination area from a captured image other than the original image (step S42, YES), it performs a replacement process to replace the captured image in which the destination area was detected with the original image (step S43). When the processor 21 replaces the original image, it recognizes the destination character string on the replaced original image by character recognition processing of a second algorithm different from the character recognition in the control device 8 (step S44). The processor 21 holds the recognition result of the destination character string obtained by the recognition processing on the replaced original image as the processing result of the thumbnail processing.

[0074] When the thumbnail processing is completed, the processor 21 determines whether or not the destination information has been successfully recognized as a result of the thumbnail processing (step S46). If the destination information has been successfully recognized by the thumbnail processing (step S46, YES), the processor 21 transmits the destination information recognized by the thumbnail processing to the control device 8 (step S34), and ends the coding processing for the coding information.

[0075] Furthermore, if the recognition of the destination information by the thumbnail process fails (step S46, NO), the processor 21 displays the original image and thumbnail images of the multiple captured images on the display unit 40, and performs a keystroke input process to have the operator key-input the destination information using the operation unit 30 (step S47). When the operator keys-inputs the destination information using the operation unit 30, the processor 21 transmits the destination information input using the operation unit 30 to the control device 8 (step S34), and ends the coding process for the coding information.

[0076] As described above, the VCD as a coding device in the first embodiment obtains coding information including an original image selected by the control device of the item processing system from multiple images taken from multiple directions of the item to be processed and thumbnail images of the multiple images, and if recognition of the destination character string in the original image fails, detects the destination area from the thumbnail image, replaces the photographed image of the thumbnail image in which the destination area was detected with the original image, and performs processing to recognize the destination character string.

[0077] As a result, according to the first embodiment, even if character recognition of an original image selected by an article processing system fails, the captured image of a thumbnail image in which a destination area has been detected can be replaced with the original image for character recognition, thereby improving the final character recognition rate. As a result, the productivity of the destination information input work (coding process) by the coding device can be improved. Second embodiment Next, a second embodiment will be described. FIG. 9 is a diagram showing an example of the configuration of a recognition system 102 including a coding device according to the second embodiment. The recognition system 102 according to the second embodiment includes an article processing system 1, a VCS 2, and a second coding system 5 (5A, 5B, 5C, 5D). The article processing system 1 and the VCS 2 may have the same hardware configuration as that described in the first embodiment. Therefore, detailed descriptions of the configurations of the article processing system 1 and the VCS 2 will be omitted.

[0078] However, in the second embodiment, each VCD 20 constituting the VCS 2 does not perform character recognition processing using the second algorithm, and the second coding system 5 is connected to the operation interface 26 and the display interface 27. The operation interface 26 of the VCD 20 is connected to a keyboard (KB) / mouse emulator (also simply referred to as an emulator) 60 of the second coding system 5 described later, and receives operation signals from the keyboard / mouse emulator 60. In addition, the display interface 27 of the VCD 20 is connected to a capture board 70 of the second coding system 5 described later, and transmits image data for displaying a display screen displayed by the VCD 20 (display data for displaying the display screen on a display device) to the capture board 70.

[0079] In the second embodiment, when each VCD 20 receives coding information from the item processing system 1, it supplies a display screen displaying the original image and thumbnail image contained in the coding information to the second coding system 5 via the display interface 27, obtains destination information for the display screen as key input information from the second coding system 5 via the operation interface 26, and transmits the destination information based on the key input information obtained from the second coding system 5 to the item processing system 1.

[0080] Next, a configuration of a second coding system 5 having a second coding device 50 which is an example of a coding device according to the second embodiment will be described. Since the second coding systems 5A to 5D have the same configuration, they will be described as the second coding system 5, and since the second coding devices 50a to 50d have the same configuration, they will be described as the second coding device 5.

[0081] 9, the second coding system 5 according to the second embodiment includes a second coding device 50, a keyboard / mouse emulator (emulator) 60, a capture board 70, an operation unit 80, and a display unit 90. The second coding device 50 and the VCD 20 are connected via the keyboard (KB) / mouse emulator 60 and the capture board 70. The operation unit 80 and the display unit 90 are connected to the second coding device 50.

[0082] FIG. 10 is a block diagram showing an example of the configuration of a second coding device 50, which is an example of a coding device according to the second embodiment. As shown in FIG. 10, the second coding device 50 includes a processor 51, a ROM 52, a RAM 53, an NVM 54, a communication unit 55, an emulator interface 56, an image interface 57, an operation interface 58, and a display interface 59.

[0083] The processor 51 is connected to the ROM 52, the RAM 53, the NVM 54, the communication unit 55, the emulator interface 56, the image interface 57, the operation interface 58 and the display interface 59 via a data bus or the like. Note that the second coding device 50 may be provided with a configuration as necessary in addition to the configuration shown in Fig. 10, or a specific configuration may be excluded from the configuration shown in Fig. 10.

[0084] The processor 51 controls the overall operation of the second coding device 50. The processor 51 may include not only a processing circuit for executing a program, but also an internal cache and various interfaces. The processor 51 realizes various processes by executing a program stored in advance in the internal memory, the ROM 52, or the NVM 54. Some of the various functions realized by the processor 51 executing the programs may be realized by a hardware circuit. In this case, the processor 51 controls the functions executed by the hardware circuit.

[0085] The ROM 52 is a non-volatile memory in which control programs, control data, etc. are stored in advance. The control programs and control data stored in the ROM 52 are incorporated in advance in accordance with the specifications of the second coding device 50.

[0086] The RAM 53 is a volatile memory. The RAM 53 temporarily stores data being processed by the processor 51. The RAM 53 stores various application programs based on instructions from the processor 51. The RAM 53 may also store data necessary for executing the application programs, execution results of the application programs, and the like.

[0087] The NVM 54 is a non-volatile memory in which data can be written and rewritten. For example, the NVM 54 is composed of a HDD, an SSD, a flash memory, or the like. The NVM 54 stores a control program, an application program, and various data according to the operational purpose of the second coding device 50.

[0088] The communication unit 55 is an interface for transmitting and receiving data. For example, the communication unit 55 may include an interface for transmitting and receiving data to and from another second coding device 50.

[0089] The emulator interface 56 is connected to a keyboard / mouse emulator 60. The emulator interface 56 is an interface for transmitting and receiving data to and from the keyboard / mouse emulator 60. The emulator interface 56 causes the keyboard / mouse emulator 60 to output a control signal from the processor 51 to the VCD 20 as an operation signal in response to a key input. That is, the emulator interface 56 inputs the operation signal to the VCD 20 through the keyboard / mouse emulator 60.

[0090] The image interface 57 is an interface for transmitting and receiving data to and from the capture board 70. The image interface 57 obtains an image of the display screen of the VCD 20 captured by the capture board 70. The capture board 70 obtains an image of the display screen (display image data supplied by the VCD 20 to the display device) that the VCD 20 displays on the display device. The image interface 57 supplies the image of the display screen of the VCD 20 captured by the capture board 70 to the processor 51.

[0091] The operation interface 58 is an interface for transmitting and receiving data to and from the operation unit 80. For example, the operation interface 58 receives an operation signal indicating an operation input to the operation unit 80 from the operation unit 80. The operation interface 58 transmits the received operation signal to the processor 51. The operation interface 58 may also supply power to the operation unit 80.

[0092] The display interface 59 is an interface for transmitting and receiving data to and from the display unit 90. The display interface 59 outputs image data from the processor 51 to the display unit 90.

[0093] The emulator interface 56, the image interface 57, the operation interface 58 and the display interface 59 (or a part of them) may be integrally formed in the main body of the second coding device 50. The second coding device 50 may include one that performs the second recognition process and does not require keystroke input by an operator. The second coding device 50 that does not require keystroke input by an operator may not have the operation interface 58 and the display interface 59.

[0094] Next, a description will be given of an outline of the processing executed by the second coding device 50, which is an example of a coding device according to the second embodiment. The processor 51 of the second coding device 50 realizes various processing functions by executing programs stored in the ROM 52, the NVM 54, or the like.

[0095] First, the processor 51 acquires an image of the display screen from the VCD 20 through the image interface 57. As described in the first embodiment, the VCD 20 generates display (image) data for displaying a display screen including an original image included in the coding information from the control device 8 and thumbnail images of a plurality of captured images. For example, as shown in Fig. 5 or Fig. 7, the VCD 20 generates display data for displaying on the display device a display screen (coding screen) that displays an original image in the main display area 41 and thumbnail images of a plurality of captured images in the sub-display area 42.

[0096] The capture board 70 obtains display data of the display screen (coding screen) from the VCD 20, captures an image of the display screen displayed by the display data, and supplies the captured image data of the display screen to the second coding device 50. That is, the processor 51 of the second coding device 50 obtains the image data of the display screen of the VCD 20 captured by the capture board 70 via the image interface 57.

[0097] The processor 51 also has a function of acquiring the display position, orientation, and size of the destination area of ​​the original image displayed in the main display area 41 on the display screen of the VCD 30. The processor 51 also has a function of determining whether to input a transformation operation for the original image to the VCD 20 based on the position, orientation, and size of the destination area, and inputting an operation signal instructing the transformation operation to the VCD 20.

[0098] Here, the processor 51 detects a destination area in an original image and obtains a position, orientation, and size of the destination area. The processor 51 extracts an original image from the display screen of the VCD 20 and detects a destination area in the extracted original image. The processor 51 obtains a position, orientation, and size of the destination area in the original image according to a predetermined algorithm. For example, the NVM 54 stores in advance a model (e.g., a neural network) that outputs a position, orientation, and size of the destination area. The processor 51 obtains a position, orientation, and size of the destination area in the original image based on the model stored in the NVM 54.

[0099] The NVM 54 may store in advance a model that outputs the position of the destination area when a photographed image is input, a model that outputs the orientation of the destination area when a photographed image is input, and a model that outputs the size of the destination area when a photographed image is input. In this case, the processor 51 inputs an original image extracted from the display screen to each model to obtain the position, orientation, and size of the destination area.

[0100] Furthermore, processor 51 determines whether to input a transformation operation for the original image to VCD 20 based on the position, orientation, and size of the destination area. Processor 51 determines whether character recognition processing can be appropriately performed on the destination area of ​​the original image displayed in main display area 41 of the display screen of VCD 20.

[0101] For example, processor 51 determines whether the size of the destination area in the original image is equal to or larger than a predetermined size. If the size of the destination area is smaller than the predetermined size, processor 51 determines to input a transformation operation that makes the size of the destination area equal to or larger than the predetermined size. For example, if the size of the destination area is smaller than the predetermined size, processor 51 determines to input a transformation operation that enlarges the original image in main display area 41.

[0102] In addition, processor 51 determines whether the orientation of the destination area in the original image is upright. If it is determined that the orientation of the destination area is not upright, processor 51 determines to input a transformation operation that rotates the orientation of the destination area in the original image upright. For example, if the orientation of the destination area is tilted 90 degrees to the left, processor 51 determines to input a transformation operation that rotates the original image in main display area 41 90 degrees to the right.

[0103] Processor 51 also determines whether the destination area of ​​the original image is cut off in main display area 41 of the display screen of VCD 20. If it is determined that the destination area is cut off, processor 51 determines to input a transformation operation to fit the destination area of ​​the original image into main display area 41. For example, if the right end of the destination area is cut off, processor 51 determines to input a transformation operation to move the original image in main display area 41 to the left. If the destination area of ​​the original image does not fit into main display area 41, processor 51 determines to input a transformation operation to reduce the original image in main display area 41.

[0104] Furthermore, the processor 51 has a function of inputting an operation signal instructing the transformation operation to the VCD 20 when it is determined that a transformation operation for the original image is to be input to the VCD 20. When it is determined that a transformation operation is to be input, the processor 51 uses the keyboard / mouse emulator 26 to transmit an operation signal instructing the transformation operation determined to be input to the operation interface 26 of the VCD 20. The processor 51 causes the keyboard / mouse emulator 60 to generate an operation signal instructing the transformation operation through the emulator interface 56, and causes the keyboard / mouse emulator 60 to output the operation signal instructing the transformation operation generated to the operation interface 26 of the VCD 20. As a result, the VCD 20 outputs, through the display interface 27, a display screen in which the original image in the main display area 41 has been updated (transformed) in accordance with the transformation operation.

[0105] The processor 51 also has a function of performing a replacement process to replace the original image displayed in the main display area 41 on the display screen of the VCD 30 with another captured image. If the processor 51 fails to recognize the destination character string in the second recognition process described below for the original image specified by the control device 8, the processor 51 detects the destination area from thumbnail images of captured images other than the original image. If the destination area is detected from the thumbnail image, the processor 51 inputs an operation signal for a replacement operation to the VCD 20 to instruct the processor 51 to replace the original image displayed in the main display area 41 with the captured image in which the destination area has been detected.

[0106] The processor 51 uses the keyboard / mouse emulator 60 to transmit an operation signal for a replacement operation that instructs replacement of the original image to the operation interface 26 of the VCD 20 via the emulator interface 56. The processor 51 causes the keyboard / mouse emulator 60 connected via the emulator interface 56 to generate an operation signal for instructing the replacement operation, and causes the keyboard / mouse emulator 60 to output the operation signal for the replacement operation generated by the keyboard / mouse emulator 60 to the operation interface 26 of the VCD 20. As a result, the VCD 20 outputs, via the display interface 27, a display screen in which the original image in the main display area 41 has been replaced (updated) with the designated captured image in accordance with the replacement operation.

[0107] Processor 51 also has a function of recognizing the character string of the destination by a second recognition process for the destination area of ​​the original image. Processor 51 performs character recognition process (second recognition process) for the destination area detected from the original image on the display screen of VCD 20, recognizing the character string of the destination according to a predetermined algorithm (second algorithm) different from the first algorithm. The second algorithm is configured to recognize at least a part of the characters that cannot be recognized by the first algorithm. Processor 51 identifies the destination information of the item based on the result of the second recognition process.

[0108] When the processor 51 is able to identify the destination information by the character recognition process (second recognition process), it inputs the identified destination information to the VCD 20. The processor 51 converts the identified destination information by the keyboard / mouse emulator 60 into an operation signal by key input, and supplies it to the operation interface 26 of the VCD 20. That is, the processor 51 generates an operation signal for inputting the destination information to the keyboard / mouse emulator 4 through the emulator interface 56, and outputs the operation signal for key input of the generated destination information to the operation interface 26 of the VCD 20.

[0109] Furthermore, if the second recognition process fails to recognize the character string of the destination, the processor 51 performs a keystroke input process in which the operator keys in the destination information via the operation unit 80. If the second recognition process fails to recognize the character string of the destination, the processor 51 displays a display screen from the VCD 20 on the display unit 90, and accepts key input of the destination information via the operation unit 80. The processor 51 inputs the destination information input using the operation unit 80 to the VCD 20 as an operation signal indicating the information keyed in by the emulator 60 connected to the emulator interface 56.

[0110] Furthermore, the processor 51 acquires in real time the display screen of the VCD captured by the capture board 70 through the display interface 27, and displays the acquired display screen on the display unit 90. The operator visually checks the original image displayed in the main display area 41 of the display screen of the VCD 20 displayed on the display unit 90, and inputs destination information through the operation unit 80. When input of the destination is completed, the operator inputs an operation to the operation unit 80 to complete input of the destination information.

[0111] The second coding system 5 may include a second coding device 50 to which the operation unit 80 and the display unit 90 are not connected. In this case, the processor 51 of the second coding device 50 to which the operation unit 80 and the display unit 90 are not connected may display the display screen of the VCD 20 on the display unit 90 connected to the other second coding device 50, and input an operation signal indicating an operation input to the operation unit 80 connected to the other second coding device 50 to the VCD 20. Also, the main second coding device 50 (e.g., the second coding device 50a) or an external control device may manage the operation unit 80 used to input the destination and the display unit 90 that displays the display screen of the VCD 20.

[0112] Next, the operation of the second coding device 50 in the second coding system 5 of the recognition system 102 according to the second embodiment will be described. FIG. 11 is a flowchart for explaining a first operation example of the second coding device 50 as the coding device according to the second embodiment.

[0113] The second coding device 50 acquires the display screen output by the VCD 20 via the image interface 57 as image data of the display screen captured by the capture board 70 (step S51). The processor 21 of the second coding device 50 holds the image data of the display screen acquired via the image interface 57 in a memory such as the RAM 53 or NVM 54. Here, the description will be given on the assumption that image data of the display screen of the first display example shown in Fig. 5 has been acquired.

[0114] The processor 51 extracts an original image to be displayed in the main display area 41 of the display screen from the image data of the display screen of the VCD 20. The processor 51 performs a second recognition process on the extracted original image to recognize a character string of a destination by character recognition of a second algorithm different from the character recognition in the control device 8 (step S52).

[0115] When the processor 51 succeeds in recognizing the destination character string by the second recognition process (step S53, YES), it outputs destination information based on the destination character string obtained by the second recognition process to the emulator 60 via the emulator interface 56 (step S54). The emulator 60 generates an operation signal for keying in the destination information output from the emulator interface 56, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and ends the coding process for the coding information.

[0116] Furthermore, in parallel with the second recognition process for the original image, the processor 51 executes thumbnail processing for recognizing destination information from a captured image other than the original image included in the coding information (steps S61-S64).

[0117] As part of the thumbnail processing, the processor 51 first performs a detection process to detect a destination area in thumbnail images of captured images other than the original image (step S61). If the processor 51 cannot detect a destination area in any of the thumbnail images of the captured images (step S62, NO), the processor 51 retains, as a processing result of the thumbnail processing, a message indicating that a destination area could not be detected in the thumbnail image.

[0118] Furthermore, if the processor 51 can detect a destination area from a thumbnail image of a photographed image other than the original image (step S62, YES), the processor 51 performs a replacement process to replace the photographed image of the thumbnail image in which the destination area was detected with the original image. The processor 51 transmits a replacement operation to replace the photographed image of the thumbnail image in which the destination area was detected with the original image to the keyboard / mouse emulator 60 via the emulator interface 56 (step S63).

[0119] The keyboard / mouse emulator 60 generates an operation signal instructing a replacement operation, and transmits the generated operation signal for the replacement operation to the operation interface 26 of the VCD 20. In response to the operation signal for the replacement operation received from the operation interface 26, the processor 21 of the VCD 20 generates (updates) a display screen in which the original image in the main display area 41 is replaced with the designated captured image in response to the operation signal, and outputs image data of the generated display screen via the display interface 27.

[0120] The capture board 70 captures the display screen in which the original image output by the display interface 27 has been replaced, and transmits image data of the captured display screen to the image interface 57 of the second coding device 50. The processor 51 of the second coding device 50 acquires the display screen in which the original image has been replaced, received by the image interface 57 (step S64).

[0121] For example, in the display screen of the first display example shown in Fig. 5, the captured image displayed in the fifth area 425 of the sub-display area 42 is displayed as an original image in the main display area 41. When a destination area is detected in the thumbnail image displayed in the first display area 421 of the sub-display area 42 in the display screen of the first display example shown in Fig. 5, the processor 51 transmits an operation signal for a replacement operation to replace the captured image of the thumbnail image displayed in the first display area 421 with the original image to the keyboard / mouse emulator 60 via the emulator interface 56. The keyboard / mouse emulator 60 generates an operation signal for instructing to replace the captured image of the thumbnail image in the first display area 421 with the original image, and transmits the generated operation signal to the operation interface 26 of the VCD 20.

[0122] Processor 21 of VCD 20 receives an operation signal from operation interface 26 instructing to replace the captured image in first display area 421 with an original image. Processor 21 of VCD 20 obtains from control device 8 a full-size captured image of the thumbnail image (thumbnail image designated for replacement) displayed in first display area 421.

[0123] Note that full-size image data of the multiple captured images displayed as thumbnail images in the sub-display area 42 may be received as coding information from the control device 8 and stored in a memory such as the RAM 23. In this case, the processor 21 of the VCD 20 may obtain full-size captured images of the thumbnail images displayed in the first display area 421 from the memory.

[0124] When the processor 21 of the VCD 20 acquires a full-size photographed image of a thumbnail image for which replacement has been specified, the processor 21 generates (updates) a display screen as shown in Fig. 7 by displaying the acquired full-size photographed image as an original image in the main display area 41. When the processor 21 of the VCD generates (updates) a display screen in which the original image in the main display area 41 has been replaced with the photographed image in the first display area 421, the processor 21 of the VCD outputs display data of the generated display screen via the display interface 27.

[0125] The capture board 70 captures an image of the display screen in which the original image output by the VCD 20 has been replaced with the image of the first display area 421, and transmits the image data of the captured display screen to the image interface 57 of the second coding device 50. As a result, the processor 51 of the second coding device 50 obtains a display screen as shown in FIG. 7 in which the original image received by the image interface 57 has been replaced.

[0126] When the processor 21 obtains the display screen in which the original image has been replaced, the processor 21 performs an enlargement / rotation operation (transformation operation) of the original image on the display screen obtained by the replacement process (step S65). As described above, the processor 21 detects the destination area in the original image, and determines whether to input the transformation operation on the original image to the VCD 20 based on the position, orientation, and size of the destination area.

[0127] If the processor 51 determines that a transformation operation is not to be input (a transformation operation is not required), the process proceeds to step S66. On the other hand, if the processor 51 determines that a transformation operation is to be input, the processor 51 generates a transformation instruction for instructing the movement, enlargement, rotation, etc. of the position of the original image, and supplies an operation signal for the transformation operation to the VCD 20 via the emulator interface 56 and the emulator 60.

[0128] The VCD 20 generates a display screen for displaying a deformed (updated) original image in the main display area 41 in response to an operation signal for a transformation operation input to the operation interface 26, and outputs the display screen from the display interface 27. The display screen for displaying the deformed original image is captured by the capture board 70, and image data of the captured display screen is supplied to the image interface 57 of the second coding device 50. As a result, the processor 51 of the second coding device 50 obtains the display screen for displaying the original image transformed in response to the transformation operation.

[0129] When processor 51 acquires the display screen in which the original image has been replaced or the display screen in which the replaced original image has been transformed, processor 51 recognizes the destination character string for the original image on the acquired display screen by character recognition processing of a second algorithm different from the character recognition of control device 8 (step S66). Processor 51 holds the recognition result of the destination character string for the replaced original image by character recognition using the second algorithm as the processing result of thumbnail processing.

[0130] If the processor 51 cannot recognize the destination character string from the replaced original image, the processor 51 stores the fact that the destination information could not be recognized from the captured image of the thumbnail image as the processing result of the thumbnail processing. Also, if the processor 51 can recognize the destination character string from the replaced original image, the processor 51 stores the destination information including the recognized destination character string as the processing result of the thumbnail processing.

[0131] On the other hand, if processor 51 fails to recognize the destination character string for the original image designated by control device 8 (step S53, NO), processor 51 determines whether or not the thumbnail processing as described above has ended (step S55). If the thumbnail processing has not ended (step S55, NO), processor 51 waits for the thumbnail processing to end.

[0132] When the thumbnail processing is completed (step S55, YES), the processor 51 judges whether or not the destination information has been successfully recognized as a result of the thumbnail processing (step S56). When the destination information has been successfully recognized by the thumbnail processing (step S56, YES), the processor 51 supplies the destination information recognized by the thumbnail processing to the emulator 60 via the emulator interface 56 (step S54). The emulator 60 generates an operation signal for keying in the destination information received from the emulator interface 56 of the second coding device 50, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and ends the coding process for the coding information.

[0133] Furthermore, if the recognition of the destination information by the thumbnail processing fails (step S56, NO), the processor 51 displays a display screen such as that shown in FIG. 5 or 7 acquired by the image interface 57 on the display unit 90, and performs a keystroke input process to have the operator use the operation unit 80 to key in the destination information (step S57).

[0134] The operator keys in the destination information through the operation unit 80 while visually checking (referring to) the display screen shown in FIG. 5 or 7 displayed on the display unit 90. The processor 51 of the second coding device 50 acquires the destination information that the operator keys in through the operation unit 80. When the processor 21 acquires the destination information that the operator keys in through the operation unit 80, it supplies the acquired destination information to the emulator 60 through the emulator interface 56 (step S54). The emulator 60 generates an operation signal for keying in the destination information received from the emulator interface 56 of the second coding device 50, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and ends the coding process for the coding information.

[0135] Next, a second operation example of the second coding device 50 as a coding device according to the second embodiment will be described. In the second operation example, the thumbnail processing executed as parallel processing in the first operation example is executed when recognition of the destination character string for the original image designated by the control device 8 fails. FIG. 12 is a flowchart for explaining a second operation example of the second coding device 50 as the coding device according to the second embodiment.

[0136] The second coding device 50 acquires the display screen output by the VCD 20 via the image interface 57 as image data of the display screen captured by the capture board 70 (step S71). The processor 21 of the second coding device 50 holds the image data of the display screen acquired via the image interface 57 in a memory such as the RAM 53 or NVM 54. Here, the description will be given on the assumption that image data of the display screen of the first display example shown in Fig. 5 has been acquired.

[0137] The processor 51 extracts an original image to be displayed in the main display area 41 of the display screen from the image data of the display screen of the VCD 20. The processor 51 performs a second recognition process on the extracted original image to recognize a character string of a destination by character recognition of a second algorithm different from the character recognition in the control device 8 (step S72).

[0138] When the processor 51 succeeds in recognizing the destination character string by the second recognition process (step S73, YES), it outputs destination information based on the destination character string obtained by the second recognition process to the emulator 60 via the emulator interface 56 (step S74). The emulator 60 generates an operation signal for keying in the destination information received from the emulator interface 56, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and ends the coding process for the coding information.

[0139] In addition, if the processor 51 fails to recognize the destination character string using the second recognition process for the original image specified by the control device 8 (step S73, NO), it executes thumbnail processing to recognize destination information from a captured image other than the original image displayed as a thumbnail image (steps S81-S86).

[0140] As thumbnail processing, processor 51 performs detection processing to detect a destination area in thumbnail images of captured images other than the original image (step S81). If processor 51 cannot detect a destination area in any thumbnail image of the captured image (step S82, NO), processor 51 retains a result of thumbnail processing indicating that a destination area could not be detected in the thumbnail image.

[0141] Furthermore, when the processor 51 detects a destination area from a thumbnail image of a photographed image other than the original image (step S82, YES), it transmits a replacement operation to replace the photographed image of the thumbnail image in which the destination area was detected with the original image to the keyboard / mouse emulator 60 via the emulator interface 56 (step S83). As a result, the keyboard / mouse emulator 60 transmits an operation signal instructing the replacement operation to the operation interface 26 of the VCD 20. The processor 21 of the VCD 20 generates (updates) a display screen in which the original image in the main display area 41 has been replaced with the designated photographed image in response to the operation signal of the replacement operation received via the operation interface 26, and outputs image data of the generated display screen via the display interface 27.

[0142] The capture board 70 captures the display screen in which the original image output by the display interface 27 has been replaced, and transmits image data of the captured display screen to the image interface 57 of the second coding device 50. As a result, the processor 51 of the second coding device 50 acquires the display screen in which the original image has been replaced, received by the image interface 57 (step S84).

[0143] Furthermore, when the processor 21 obtains the display screen in which the original image has been replaced, it performs an enlargement / rotation operation (transformation operation) of the original image on the display screen obtained by the replacement process (step S85). If the processor 51 determines that a transformation operation is not to be input (transformation operation is not necessary), the process proceeds to step S66. If the processor 51 determines that a transformation operation is to be input, the processor 51 generates a transformation instruction for instructing the movement, enlargement, rotation, etc. of the position of the original image, and supplies an operation signal for the transformation operation to the VCD 20 via the emulator interface 56 and the emulator 60.

[0144] The VCD 20 generates a display screen for displaying a deformed (updated) original image in the main display area 41 in response to an operation signal for a transformation operation input to the operation interface 26, and outputs the display screen from the display interface 27. The display screen for displaying the deformed original image is captured by the capture board 70, and image data of the captured display screen is supplied to the image interface 57 of the second coding device 50. As a result, the processor 51 of the second coding device 50 obtains the display screen for displaying the original image transformed in response to the transformation operation.

[0145] When processor 51 acquires the display screen in which the original image has been replaced or the display screen in which the replaced original image has been transformed, processor 51 recognizes the destination character string for the original image on the acquired display screen by character recognition processing of a second algorithm different from the character recognition of control device 8 (step S86). Processor 51 holds the recognition result of the destination character string for the replaced original image by character recognition using the second algorithm as the processing result of thumbnail processing.

[0146] If the processor 51 cannot recognize the destination character string from the replaced original image, the processor 51 stores the fact that the destination information could not be recognized from the captured image of the thumbnail image as the processing result of the thumbnail processing. Also, if the processor 51 can recognize the destination character string from the replaced original image, the processor 51 stores the destination information including the recognized destination character string as the processing result of the thumbnail processing.

[0147] The processor 51 judges whether or not the recognition of the destination information is successful as a result of the thumbnail processing (step S87). If the recognition of the destination information by the thumbnail processing is successful (step S87, YES), the processor 51 supplies the destination information recognized by the thumbnail processing to the emulator 60 via the emulator interface 56 (step S74). The emulator 60 generates an operation signal for keying in the destination information received from the emulator interface 56 of the second coding device 50, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and ends the coding process for the coding information.

[0148] Furthermore, if the recognition of the destination information by the thumbnail processing fails (step S87, NO), the processor 51 displays a display screen such as that shown in FIG. 5 or FIG. 7 acquired by the image interface 57 on the display unit 90, and performs a keystroke input process to have the operator use the operation unit 80 to key in the destination information (step S89).

[0149] When the processor 51 acquires the destination information keyed in by the operator using the operation unit 80, it supplies the acquired destination information to the emulator 60 via the emulator interface 56 (step S74). The emulator 60 generates an operation signal for keying in the destination information received from the emulator interface 56 of the second coding device 50, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and ends the coding process for the coding information.

[0150] As described above, the second coding device as the coding device according to the second embodiment is connected to the display interface and the operation interface of the VCD in the existing VCS. The second coding device acquires an image of the display screen of the existing VCD including an original image selected from a plurality of photographed images of the article to be processed and thumbnail images of the plurality of photographed images. When the second coding device fails to recognize the destination character string in the original image extracted from the display screen acquired from the existing VCD, the second coding device supplies an operation signal for an exchange operation to the existing VCD for replacing the photographed image of the thumbnail image in which the destination area has been detected with the original image. The second coding device acquires the display screen in which the original image has been replaced from the existing VCD and performs a process of recognizing the destination character string in the replaced original image.

[0151] As a result, even if character recognition of the original image on the display screen acquired from the VCD fails, if the destination area can be detected from the thumbnail image of the display screen, the captured image of the thumbnail image in which the destination area was detected can be replaced with the original image for character recognition, thereby improving the final character recognition rate. As a result, it is possible to improve the recognition rate of destination information by the second coding device and improve the productivity of the coding process without changing the configuration of the existing VCD.

[0152] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0153] 1...article processing system, 2...VCS, 5 (5A, 5B, 5C, 5D)...second coding system, 6...sorter, 7...camera, 8...control device, 11...processor, 12...ROM, 13...RAM, 14...NVM, 15...camera interface, 16...communication unit, 19...sorter interface, 20 (20a, 20b, 20c, 20d)...VCD (coding device), 21...processor, 22...ROM, 23...RAM, 24...NVM, 25...communication unit, 26...operation interface, 27...display interface, 30...operation unit, 40...display unit, 50 (50 a, 50b, 50c, 50d)...second coding device (coding device), 51...processor, 52...ROM, 53...RAM, 54...NVM, 55...communication unit, 56...emulator interface, 57...image interface, 58...operation interface, 59...display interface, 60 (60a, 60b, 60c, 60d)...keyboard (KB) / mouse emulator, 70 (70a, 70b, 70c, 70d)...capture board, 80 (80a, 80b, 80c, 80d)...operation unit, 90 (90a, 90b, 90c, 90d)...display unit.

Claims

1. a communication unit that acquires an original image selected from a plurality of photographed images of an article and thumbnail images of the plurality of photographed images; If the recognition of the destination character string for the original image fails, a destination area is detected from a thumbnail image of a photographed image other than the original image; a process of replacing the photographed image of the thumbnail image in which the destination area is detected with the original image and recognizing the character string of the destination; A processor; A coding device having the following construction.

2. The communication unit receives, as an original image, a captured image in which a control device of an article processing system that processes the article fails to recognize a character string of a destination; the processor recognizes the destination character string for the original image by character recognition processing according to a second algorithm different from the recognition of the destination character string by the control device; 2. The coding device according to claim 1.

3. an operation interface connected to the operation unit; A display interface connected to the display unit, When the processor fails to recognize a character string of a destination for the original image, the processor displays the original image and thumbnail images of the plurality of captured images on the display unit, and transmits destination information inputted to the operation unit to the control device.

3. Coding device according to claim 2.

4. an image interface for acquiring image data of a display screen to be displayed by another coding device; an emulator interface connected to an emulator that generates an operation signal to be input to the other coding device; acquiring image data for a display screen of the other coding device that displays an original image selected from a plurality of photographed images of an article by the image interface and thumbnail images of the plurality of photographed images; If the recognition of the destination character string for the original image on the display screen fails, a destination area is detected from a thumbnail image of a photographed image other than the original image; a swapping operation for swapping the captured image of the thumbnail image in which the destination area is detected with the original image is transmitted to the emulator through the emulator interface, and then the other coding device acquires, through the image interface, image data of a display screen in which the captured image that becomes the original image has been swapped in accordance with the swapping operation; performing character recognition to recognize a character string of a destination for the original image after the replacement on the display screen; A processor; A coding device having the following construction.

5. The processor, sending a transformation operation to an emulator via the emulator interface for transforming the original image according to a state of a destination area in the original image after the swap on the display screen; obtaining image data of a display screen for displaying an original image transformed by the transformation operation performed by the other coding device through the image interface; performing character recognition to recognize a character string of a destination address for the transformed original image on the display screen; 5. Coding device according to claim 4.

6. the image interface is connected to a display interface of another coding device connected to a control device of an article processing system for processing the article; The emulator interface is connected to an operation interface of the other coding device via an emulator; the processor recognizes the destination character string for the original image by character recognition processing according to a second algorithm different from the recognition of the destination character string by the control device; Coding device according to any one of claims 4 and 5.

7. an operation interface connected to the operation unit; A display interface connected to the display unit, when the processor fails to recognize a destination character string on a display screen generated by the other coding device and acquired through the image interface, the processor displays the display screen on the display unit, and transmits destination information inputted on the operation unit to the emulator through the emulator interface; Coding device according to claim 6.

8. On the computer, A function of acquiring an original image selected from a plurality of photographed images of an article and a thumbnail image of the plurality of photographed images; a function of detecting a destination area from a thumbnail image of a photographed image other than the original image when the recognition of the destination character string for the original image fails; a function of replacing the captured image of the thumbnail image in which the destination area is detected with an original image and performing a process of recognizing the character string of the destination; A coding processing program that realizes this.

9. a computer including an image interface for acquiring image data of a display screen displayed by another coding device, and an emulator interface for connecting to an emulator for generating an operation signal to be input to the other coding device, a function of acquiring image data for a display screen of the other coding device that displays an original image selected from a plurality of images of an article captured by the image interface and thumbnail images of the plurality of images; a function of detecting a destination area from a thumbnail image of a photographed image other than the original image when the recognition of a destination character string for the original image on the display screen fails; a function of transmitting a replacement operation for replacing the captured image of the thumbnail image in which the destination area is detected with the original image to an emulator through the emulator interface, and then acquiring, through the image interface, image data of a display screen in which the captured image that becomes the original image is replaced by the other coding device in accordance with the replacement operation; a function of performing character recognition to recognize a character string of a destination for the original image after the replacement on the display screen; A coding processing program that realizes this.

Images