Coding device and coding processing program

Abstract

To provide a coding device and a coding processing program capable of improving a recognition rate of a character string.SOLUTION: According to an embodiment, a coding device has a communication unit and a processor. The communication unit acquires an original image selected from a plurality of captured images obtained by photographing an article and thumbnail images of the plurality of captured images. If the processor fails to recognize a character string of a destination for the original image, the processor detects a destination area from the thumbnail images of the captured images other than the original image, replaces the captured image of the thumbnail image in which the destination area has been detected with the original image, and performs processing of recognizing the character string of the destination.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] Embodiments of the present invention relate to a coding device and a coding processing program.

Background Art

[0002] Conventionally, there is a coding device called a video coding disk (VCD) for identifying a destination for an image in which the destination string cannot be recognized in an article processing system. In recent years, there is also an article processing system that captures an article to be processed from a plurality of directions, detects a captured image (original image) including a destination string from a plurality of captured images, and performs character recognition. When such an article processing system cannot recognize the destination string, it supplies, together with the original image for which the destination string is to be recognized, captured images other than the original image as thumbnail images to the VCD.

[0003] A conventional VCD performs character recognition on an original image from an article processing system using an algorithm different from that of the article processing system to recognize the destination string, and when the recognition fails, it causes the operator to input the destination. However, it is possible that there is no destination in the original image detected by the article processing system and there is a description of the destination in the captured image as a thumbnail image. In the conventional VCD, when the operator discovers a thumbnail image with a destination string, the operator can specify the thumbnail image with the destination. In this case, the VCD accepts the destination input by the operator in a state where the captured image of the specified thumbnail image is replaced with the original image and displayed. That is, the conventional VCD has a problem that the workload of the operator increases when there is no destination in the original image and there is a destination in the thumbnail image.

[0004] Furthermore, in recent years, a system has been proposed in which a second coding device is connected to an existing VCD, and the destination string is identified by character recognition by the second coding device or by keystroke input by an operator. The second coding device captures the image displayed on the display unit of the existing VCD and performs character recognition on the captured image, which presents a problem in that the operator's workload increases when the destination is not present in the original image but is in the thumbnail image. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2021-140632 [Overview of the project] [Problems that the invention aims to solve]

[0006] To solve the above problems, we provide a coding device and a coding processing program that can improve the recognition rate of character strings. [Means for solving the problem]

[0007] According to the embodiment, the coding device has a communication unit and a processor. The communication unit communicates with an article processing system that processes articles, and from a plurality of captured images of the articles taken from multiple directions The control device of the aforementioned article processing system Selected 1 Original image and the aforementioned 1 The processor receives thumbnail images of captured images other than the original image from the item processing system. 1 A recognition process that recognizes the destination string for the original image, and the 1 The destination region is detected from the thumbnail images of captured images other than the original image, and the thumbnail image in which the destination region was detected is then selected. The corresponding captured image is set as the second original image, and for this second original image... Perform thumbnail processing that recognizes the destination string. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a block diagram showing an example configuration of a recognition system including a coding device according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the configuration of a control device for an item processing system included in the recognition system according to the first embodiment. [Figure 3] Figure 3 is a block diagram showing an example configuration of a VCD as a coding device according to the first embodiment. [Figure 4] Figure 4 is a flowchart illustrating a first example of operation using the VCD as a coding device according to the first embodiment. [Figure 5] Figure 5 shows a first example of the display screen displayed on the display unit of the VCD, which is a coding device according to the first embodiment. [Figure 6] Figure 6 shows an example of the relationship between a thumbnail image displayed on the display screen by the coding device according to the first embodiment and multiple captured images. [Figure 7] Figure 7 shows a second example of the display screen displayed on the display unit of the VCD, which is a coding device according to the first embodiment. [Figure 8] Figure 8 is a flowchart illustrating a second example of operation using the VCD as a coding device according to the first embodiment. [Figure 9] Figure 9 shows an example configuration of a recognition system including a coding device according to the second embodiment. [Figure 10] Figure 10 is a block diagram showing an example configuration of a second coding device as a coding device according to the second embodiment. [Figure 11] Figure 11 is a flowchart illustrating a first example of operation of the second coding device as a coding device according to the second embodiment. [Figure 12] Figure 12 is a flowchart illustrating a second example of operation of the second coding device as a coding device according to the second embodiment. [Modes for carrying out the invention]

[0009] Each embodiment will be described below with reference to the drawings. (First Embodiment) First, the first embodiment will be described. Figure 1 shows an example configuration of a recognition system 101 including a coding device according to the first embodiment. The recognition system 101 according to the first embodiment includes an item processing system 1 and a VCS (video coding system) 2. The item processing system 1 is a system that processes an item based on destination information displayed at any point on the item, which is a three-dimensional object. The VCS 2 is a system that identifies destination information contained in the image from the item processing system 1 and returns the identified destination information to the item processing system 1.

[0010] In the configuration example shown in Figure 1, the item processing system 1 includes a sorter 6, a camera 7, and a control device 8. The control device 8 is connected to the sorter 6 and the camera 7 and is connected to the VCS2 in a communicative manner. The control device 8 has a function to perform character recognition processing to recognize a string of characters that will be destination information. The control device 8 uses the image captured by the camera 7 to identify the recognition device as destination information for character recognition. Sorter 6 is an example of a processing device for processing items. Sorter 6 performs sorting of items to be processed based on control signals from control device 8. For example, sorter 6 sorts items supplied to a predetermined location based on control signals from control device 8 to sorting destinations according to the destination information written on the items. Sorter 6 sorts each item to the sorting destination specified by control device 8. For example, sorter 6 obtains sorting information from control device 8, which includes an ID that identifies the item and information indicating the sorting destination of the item (e.g., a chute number). Sorter 6 processes the items based on the sorting information obtained from control device 8.

[0011] Camera 7 photographs the articles to be processed by sorter 6. In the present embodiment, assume that the articles to be processed are three-dimensional, and a string of destination information is described (displayed) on any one surface of the three-dimensional object forming the article. Camera 7 is configured to photograph a three-dimensional article from multiple directions. Camera 7 is installed to acquire a plurality of captured images of the article to be processed by sorter 6 from multiple directions. Camera 7 may acquire captured images from multiple directions using multiple cameras, or may acquire a plurality of captured images by moving one camera and photographing from multiple directions. Camera 7 transmits the plurality of captured images taken to control device 8.

[0012] In the present embodiment, as an article to be processed, assume an article where destination information is described on a form in a predetermined format and pasted on the surface of the article. However, the article to be processed only needs to have the characters as destination information displayed anywhere on the surface, and may be an article with any form of form with destination information pasted thereon, or may be an article with the destination display described or printed on the surface of the article.

[0013] Control device 8 controls article processing system 1. Control device 8 acquires a plurality of captured images of the article to be processed taken by camera 7 from multiple directions. Control device 8 detects, as an original image, a captured image including the string of destination information from the plurality of captured images. Control device 8 performs character recognition processing to recognize the string of destination information on the original image detected from the plurality of captured images.

[0014] When the control device 8 fails to recognize the destination information, it transmits the original image of the article and the thumbnail image of the captured image other than the original image to the VCS 2, and acquires the destination information of the article from the VCS 2. The control device 8 sets the sorting destination of the article in the sorter 6 based on the recognized destination information or the destination information acquired from the VCS 2. For example, the control device 8 transmits sorting information including an ID for identifying an article and information indicating the sorting destination of the article to the sorter 6. Thereby, the control device 8 controls the processing of the article based on the destination information.

[0015] Also, in the configuration example shown in FIG. 1, the VCS 2 has a plurality of VCDs 20 (20a, 20b, 20c, 20d). Each VCD 20 (20a, 20b, 20c, 20d) is connected to an operation unit 30 (30a, 30b, 30c, 30d) and a display unit 40 (40a, 40b, 40c, 40d), respectively. The operation unit 30 is an operating device having operation keys for an operator to input destination information and the like. The display unit 40 is a display device that displays an image of an article that is the target of input of destination information by the operation unit 30.

[0016] Each VCD 20 performs character recognition processing (OCR processing) on the captured image supplied from the article processing system 1. In the present embodiment, each VCD 20 acquires an original image that is one of a plurality of captured images of an article captured by the control device 8 of the article processing system 1 and a thumbnail image of a captured image other than the original image. The original image is an image determined by the control device 8 to include a destination character string (an image targeted for character recognition by the control device 8), and the thumbnail image is a captured image other than the original image.

[0017] The VCD 20 executes character recognition (second character recognition) using an algorithm different from that of the control device 8 on the original image acquired from the control device 8. When the VCD 20 can recognize the destination character string by the second character recognition for the original image, the VCD 20 transmits the destination information that is the recognition result of the second character recognition to the control device 8.

[0018] Furthermore, if the second character recognition fails to recognize the destination string, the VCD20 performs a process to detect the area containing the destination string (destination area) in the thumbnail image. If the VCD20 can detect the destination area from the thumbnail image, it replaces the thumbnail image in which the destination area was detected with the original image, and performs character recognition on the image in which the destination area was detected as the original image. If the VCD20 can recognize the destination string using character recognition on the replaced original image, it transmits the destination information, which is the recognition result of the character recognition, to the control device 8.

[0019] If the destination string cannot be recognized even after replacing the original image, or if the destination area cannot be detected from the thumbnail image, the VCD20 displays the image acquired from the control device 8 on the display unit 40 and prompts the operator to key-input destination information using the operation unit 30. When the operator key-inputs destination information using the operation unit 30, the VCD20 transmits the destination information entered by the operator to the control device 8.

[0020] Next, the configuration of the control system in the control device 8 of the item processing system 1 in the recognition system 101 according to the first embodiment will be described. Figure 2 is a block diagram showing an example of the control system configuration in the control device 8 of the item processing system 1 in the recognition system 101 according to the first embodiment. As shown in Figure 2, the control device 8 includes a processor 11, ROM 12, RAM 13, NVM 14, camera interface (I / F) 15, communication unit 16, operation unit 17 and display unit 18, sorter interface (I / F) 19, and the like.

[0021] The processor 11 is connected to the ROM 12, RAM 13, NVM 14, camera interface 15, communication unit 16, operation unit 17, display unit 18, and sorter interface 19 via a data bus and interfaces. In addition to the configuration shown in Figure 2, the control device 8 may have further configurations as needed, or certain configurations may be excluded from the control device 8.

[0022] The processor 11 has the function of controlling the operation of the entire control unit 8. The processor 11 may also be equipped with an internal cache and various interfaces. The processor 11 performs various processes by executing programs that are pre-stored in the internal memory, ROM 12, or NVM 14. Furthermore, some of the various functions realized by the execution of a program by the processor 11 may be realized by hardware circuits. In this case, the processor 11 controls the functions executed by the hardware circuits.

[0023] ROM12 is a non-volatile memory in which control programs and control data are pre-stored. The control programs and control data stored in ROM12 are pre-loaded according to the specifications of the control device 8.

[0024] RAM13 is volatile memory. RAM13 temporarily stores data being processed by processor 11. RAM13 stores various application programs based on instructions from processor 11. RAM13 may also store data necessary for the execution of application programs and the execution results of application programs.

[0025] NVM14 is a non-volatile memory that allows data to be written to and rewritten. For example, NVM14 can be composed of an HDD, SSD, or flash memory. NVM14 stores control programs, applications, and various data according to the operational use of the control device 8.

[0026] The camera interface 15 is an interface for sending and receiving data with the camera 7. The camera interface 15 is an interface for communicating with the camera 7 via wired or wireless connection. The camera interface 15 receives images captured by the camera 7. The camera interface 15 transmits the received images to the processor 11. The camera interface 15 may also supply power to the camera 7.

[0027] In this embodiment, the camera 7 photographs the items to be processed by the sorter 6 from multiple directions. Therefore, the camera 7 may consist of multiple camera devices (imaging units) that photograph items at predetermined positions from multiple directions. If the camera 7 consists of multiple camera devices, the camera interface 15 shall include an interface for communication connection to the multiple camera devices acting as the camera 7.

[0028] The communication unit 16 is an interface for sending and receiving data with each VCD20 in the VCS2. For example, the communication unit 16 supports LAN (Local Area Network) connectivity. Alternatively, the communication unit 16 may support USB (Universal Serial Bus) connectivity. The communication unit 16 only needs to have an interface for sending and receiving data with the control device 8, and may also be an interface for connecting to communication control devices such as routers or switching hubs for connecting to multiple VCD20s.

[0029] The control unit 17 receives various operation inputs from the operator. The control unit 17 transmits a signal indicating the input operation to the processor 11. The control unit 17 consists of a keyboard, buttons, or a touch panel.

[0030] The display unit 18 displays information based on control from the processor 11. For example, the display unit 18 is composed of a liquid crystal monitor. If the operation unit 17 is composed of a touch panel, the display unit 18 is composed of a liquid crystal monitor integrally formed with the operation unit 17.

[0031] The sorter interface 19 is an interface for sending and receiving data with the sorter 6. For example, the sorter interface 19 communicates with the sorter 6 via wired or wireless connection. The sorter interface 19 supplies sorting information to the sorter 6 based on destination information for the items to be processed by the sorter 6. The sorter interface 19 may also receive information from the sorter 6 indicating the processing status at the sorter 6. The sorter interface 19 may also be included in the communication unit 16.

[0032] Next, the configuration of VCD20 in VCD2 as a coding device according to the first embodiment will be described. Figure 3 is a block diagram showing an example of the control system configuration in the VCD20 as a coding device according to the first embodiment. Hereinafter, VCD20a to 20d can be realized with a similar configuration and will be described as VCD20.

[0033] As shown in Figure 3, the VCD20 includes a processor 21, ROM 22, RAM 23, NVM 24, communication unit 25, operation interface (I / F) 26, and display interface (I / F) 27. The processor 21 is connected to the ROM 22, RAM 23, NVM 24, communication unit 25, operation interface 26, and display interface 27 via a data bus or the like. In addition to the configuration shown in Figure 3, the VCD20 may have other necessary configurations, or certain configurations may be excluded from the configuration shown in Figure 3.

[0034] The processor 21 controls the overall operation of the VCD 20. In addition to operators for executing programs, the processor 21 may also include an internal cache and various interfaces. The processor 21 performs various processes by executing programs stored in the internal memory, ROM 22, or NVM 24. Furthermore, some of the various functions realized by the execution of a program by the processor 21 may be realized by hardware circuits. In this case, the processor 21 controls the functions executed by the hardware circuits.

[0035] ROM22 is a non-volatile memory that stores control programs and control data. The control programs and control data stored in ROM22 are incorporated according to the specifications of VCD20.

[0036] RAM23 is a volatile memory. RAM23 temporarily stores data being processed by the processor 21. RAM23 stores various application programs based on instructions from the processor 21. RAM23 may also store data necessary for the execution of application programs and the execution results of application programs. Furthermore, RAM23 may be used as a buffer memory for temporarily storing data for transmission and reception.

[0037] NVM24 is a non-volatile memory that allows data to be written to and rewritten. For example, NVM24 can be composed of HDDs, SSDs, or flash memory. NVM24 stores control programs, applications, and various data depending on the operational use of the VCD20.

[0038] The communication unit 25 is a communication interface for sending and receiving data with the control device 8 and other components. For example, the communication unit 25 supports LAN (Local Area Network) connectivity via wired or wireless communication. Alternatively, the communication unit 25 may also support USB (Universal Serial Bus) connectivity.

[0039] The operation interface 26 is an interface for connecting to the operation unit 30. For example, the operation interface 26 receives operation signals indicating operations input to the operation unit 30, which consists of a keyboard or mouse operated by an operator. The operation interface 26 also supplies the operation signals received from the operation unit 30 to the processor 21.

[0040] The display interface 27 is an interface for connecting to the display unit 40, which displays a screen for the operator to view. The display interface 27 is an interface that outputs image data indicating the display screen to be displayed on the display unit 40 in accordance with the control of the processor 21.

[0041] Next, the operation of the control device 8 of the item processing system 1 in the recognition system 101 according to the first embodiment will be described. The control device 8 performs various processes to control the operation of the item processing system 1 by having the processor 11 execute a program stored in the ROM 12 or NVM 14, etc. In this embodiment, the processor 11 of the control device 8 has a character recognition function that recognizes the destination string in the captured image taken by the camera 7.

[0042] When the item processing system 1 processes one item, the processor 11 of the control device 8 acquires multiple images of the item to be processed, taken by the camera 7 from multiple directions. Here, the camera 7 takes pictures of the item at a predetermined shooting position (shooting area) from multiple directions.

[0043] For example, camera 7 is composed of multiple camera devices installed at multiple positions so that an object at a predetermined shooting position can be photographed from multiple directions. Alternatively, camera 7 may be composed of one or more camera devices that photograph an object from multiple directions while moving relative to the object in a predetermined shooting area. The processor 11 of the control device 8 according to this first embodiment acquires multiple captured images of a single object to be processed by camera 7, taken from multiple directions, via the camera interface 15.

[0044] When the processor 11 acquires multiple images captured by the camera 7, it performs a recognition process (first recognition process) to recognize the destination from the captured images by character recognition processing using a predetermined algorithm (first algorithm). In the first authentication process, the processor 11 performs a destination region detection process to detect the image containing the region of the destination string from the multiple captured images. The processor 11 selects the image in which the destination region was detected as the original image and performs a process to recognize the destination string on the original image. If the processor 11 succeeds in recognizing the destination string through the first recognition process, it identifies the string obtained as a result of character recognition as the destination information for the item.

[0045] Furthermore, if the first recognition process fails to recognize the destination string, the processor 11 performs a process to identify the destination information using the VCS2. That is, if the first recognition process fails to recognize the destination string, the processor 11 sends coding information, including the original image and thumbnail images of each captured image, to the VCS2 via the communication unit 16. The coding information is information that requests the VCD20 in the VCS2 to identify (code) the destination information, and includes an image for coding processing to recognize or input the destination of the item to be processed.

[0046] The image for coding includes one image (original image) selected from multiple images of the item to be processed, and thumbnail images of each image. The original image is full-size (e.g., shooting size) image data used for character recognition processing, and the thumbnail images are image data smaller than the full size for reduced display. However, the control device 8 may also send the VCD 20 the full-size image data of the multiple images and information indicating the original image in the multiple images as the image for coding. In this case, the VCD 20's processor 21 should generate the thumbnail images of the multiple images.

[0047] If the processor 11 of the control device 8 cannot recognize destination information from the captured image of the item, it selects one VCD20 from VCD20a to 20d and sends coding information, including an image for coding processing, to the selected VCD20. In response, the VCD20 identifies the destination information from the image for coding processing included in the coding information and returns the identified destination information to the control device 8. As a result, the processor 11 of the control device 8 obtains the destination information identified by the VCD20 through the communication unit 16.

[0048] The processor 11 sets the sorting destination for the item based on the destination information identified by the first recognition process or the destination information obtained from the VCD 20. For example, the processor 11 sets the sorting location to be used as the sorting destination in the sorter 6 based on the destination information. For example, the processor 11 selects a chute from among the multiple chutes (sorting locations) into which items are fed in the sorter 6 that corresponds to the administrative district (prefecture or municipality, etc.) of the destination information of the item to be processed, and sets the information indicating that chute (chute number, etc.) as the sorting destination. The processor 11 transmits sorting information, including an ID that identifies the item and information indicating the sorting destination of the item identified by the ID, to the sorter 6 via the sorter interface 19. As a result, the sorter 6 performs sorting processing to sort the items based on the sorting information supplied from the control device 8.

[0049] Next, the operation of the VCD20 in the VCS2 of the recognition system 101 according to the first embodiment will be described. The VCD20 performs destination information coding processing for multiple captured images from the control device 8 of the item processing system 1 by having the processor 21 execute a program stored in the ROM 22 or NVM 24, etc. As described above, if the first recognition processing fails, the processor 11 of the control device 8 sends coding information, including the image for coding processing of the item, to the VCD20.

[0050] Figure 4 is a flowchart illustrating a first example of operation of VDD20 as a coding device according to the first embodiment. The VCD20 receives coding information from the control device 8 via the communication unit 25, which includes multiple images of a single item taken from multiple directions (step S11). The processor 21 stores the coding information received from the control device 8 via the communication unit 25 in memory such as RAM 23 or NVM 24. The coding information includes images for coding processing that are associated with the identification information (ID) of the item. The images for coding processing include the original image, which is full-size image data, and thumbnail images of the multiple captured images.

[0051] The processor 21 performs a second recognition process on the original image extracted from the coding information, recognizing the destination string using a second character recognition algorithm different from the character recognition performed by the control device 8 (step S12). If the processor 21 successfully recognizes the destination string through the second recognition process (step S13, YES), it sends destination information generated based on the recognized destination string to the control device 8 (step S14), and terminates the processing of the coding information (coding process).

[0052] Furthermore, in parallel with the second recognition process for the original image, the processor 21 performs a thumbnail process to recognize destination information from captured images other than the original image included in the coding information (steps S21-S24). Here, it is assumed that the VCD 20 receives from the control device 8 the original image (full-size image data) and thumbnail images (thumbnail-size image data) of multiple captured images, which are designated by the control device 8 as images for coding processing included in the coding information.

[0053] The processor 21 performs a detection process as thumbnail processing to detect the area of ​​the destination string (destination area) in the thumbnail images of captured images other than the original image (step S21). For example, if it is assumed that the destination is written on a form of a predetermined format, the processor 21 detects the area of ​​the image that resembles a form of a predetermined format in each thumbnail image of captured images as the destination area. Alternatively, if it is assumed that the destination is written on the surface of an item in an arbitrary format, the processor 21 may detect the area of ​​the image of a group of characters (string group) that resembles the destination as the destination area.

[0054] If the destination region cannot be detected from any of the thumbnail images of the captured images (step S22, NO), the processor 21 stores as a result of the thumbnail processing that the destination region could not be detected from any image other than the original image specified by the control device 8.

[0055] Furthermore, if the processor 21 can detect the destination region from a captured image other than the original image (step S22, YES), it performs a replacement process to replace the captured image in which the destination region was detected with the original image (step S23). For example, if the control device 8 designates the first captured image as the original image, and the destination region is detected from the second captured image of the thumbnail image, the processor 21 obtains the full-size image data of the second captured image from the control device 8 and replaces the original image with the full-size second captured image.

[0056] The VCD20 may also receive full-size image data of multiple captured images from the control device 8 as images for coding processing included in the coding information, and store it in memory such as RAM23. In this case, the processor 21 detects the destination region from the captured images other than the original image, and if the destination region can be detected from the captured images other than the original image, the full-size image of the second captured image from which the destination information was detected should be used as the original image.

[0057] When the original image is replaced, the processor 21 recognizes the destination string on the replaced original image using a second algorithm for character recognition processing, which is different from that of the control device 8 (step S24). The processor 21 stores the recognition result of the destination string obtained by the second algorithm for character recognition processing on the replaced original image as the result of thumbnail processing.

[0058] For example, if the processor 21 cannot recognize the destination string from the replaced original image, it stores a note in the thumbnail processing result indicating that the destination information could not be recognized from any image other than the original image specified by the control device 8. Also, if the processor 21 can recognize the destination string from the replaced original image, it stores destination information including the recognized destination string in the thumbnail processing result.

[0059] On the other hand, if the processor 21 fails to recognize the destination string for the original image specified by the control device 8 (step S13, NO), it determines whether or not the thumbnail processing described above has been completed (step S15). If the thumbnail processing has not been completed (step S15, NO), the processor 21 waits for the thumbnail processing to be completed.

[0060] If the first recognition process fails and the thumbnail processing is completed (step S15, YES), the processor 21 determines whether or not the destination information was successfully recognized as a result of the thumbnail processing (step S16). If the destination information is successfully recognized by the thumbnail processing (step S16, YES), the processor 21 transmits the destination information recognized by the thumbnail processing to the control device 8 (step S14) and completes the coding process for the coding information.

[0061] Furthermore, if the recognition of destination information by thumbnail processing fails (step S16, NO), the processor 21 displays an image for coding information on the display unit 40 and performs key input processing to have the operator input destination information using the operation unit 30 (step S17). Here, the processor 21 displays the original image as the main image and the thumbnail image as a sub-image on the display screen of the display unit 40.

[0062] Figure 5 is a first display example in which the VCD20 displays a display screen including the original image and thumbnail images on the display unit 40. Figure 6 is a diagram illustrating an example of the arrangement of multiple captured images to be displayed as thumbnail images in a predetermined sub-display area 42 on the display screen. Figure 7 is a second display example of the display screen that the VCD20 displays on the display unit 40 when the original image of the display screen shown in Figure 5 is replaced.

[0063] In the display examples shown in Figures 5 and 7, the display unit 40 displays a display screen (coding screen) that includes a main display area 41, a sub-display area 42, and an input display area 43. The main display area 41 displays one image selected from multiple images of the object to be processed as the original image for character recognition processing. The sub-display area 42 has multiple display areas 421-425, each displaying thumbnail images (reduced images) of the multiple images. The input display area 43 displays information entered by the operator.

[0064] Here, as illustrated in Figure 6, the control device 8 acquires five images taken by the camera 7 from five different directions of the three-dimensional object to be processed. In this case, the control device 8 supplies the VCD 20 with an original image, which is one image selected from the five images, and thumbnail images of the five images, as images for coding. The VCD 20 displays the original image from the control device 8 in the main display area 41 and the thumbnail images of the five images in the respective display areas of the sub-display area 42.

[0065] In the display examples shown in Figures 5 to 7, the sub-display area 42 has five display areas 421, 422, 423, 424, and 425 corresponding to five captured images taken from five directions. As shown in Figure 6, the first display area 421 displays a thumbnail image of the captured image of the first surface (surface photographed from the first direction) M1 of the article. The second display area 422 displays a thumbnail image of the captured image of the second surface (surface photographed from the second direction) M2 of the article. The third display area 423 displays a thumbnail image of the captured image of the third surface (surface photographed from the third direction) M3 of the article. The fourth display area 424 displays a thumbnail image of the captured image of the fourth surface (surface photographed from the fourth direction) M4 of the article. The fifth display area 425 displays a thumbnail image of the captured image of the fifth surface (surface photographed from the fifth direction) M5 of the article.

[0066] The original image 411 displayed in the main display area 41 is, for example, the original image specified by the control device 8. Alternatively, the original image 411 displayed in the main display area 41 may be the original image replaced by thumbnail processing (the captured image in which the destination area was detected).

[0067] The original image displayed in the main display area 41 is switched according to the operator's instructions via the control unit 30. For example, in the first display example shown in Figure 5, the fifth display area 425 is selected, and the full-size image of the captured image whose thumbnail is displayed in the fifth display area 425 is displayed in the main display area 41 as the original image 411. In the display state shown in Figure 5, when the operator instructs to select a thumbnail image displayed in the first display area 421, the processor 21 selects the first display area 421, as shown in the second display example shown in Figure 7, and displays the full-size image of the captured image whose thumbnail is displayed in the first display area 421 as the original image 412 in the main display area 41.

[0068] The operator inputs destination information using the control unit 30 while visually observing (referencing) the display screen shown in Figure 5 or Figure 7. The processor 21 of the VCD20 acquires the destination information to be entered by the operator by obtaining the information entered by the operator using the control unit 30. When the operator inputs destination information using the control unit 30, the processor 21 transmits the destination information entered using the control unit 30 to the control device 8 (step S14), and terminates the coding process for the coding information.

[0069] Next, a second example of operation of VDD20 as a coding device according to the first embodiment will be described. In the second example of operation, the thumbnail processing that was performed as parallel processing in the first example of operation is performed when the recognition of the destination string for the original image specified by the control device 8 fails. Figure 8 is a flowchart illustrating a second example of operation of VDD20 as a coding device according to the first embodiment.

[0070] The VCD20 receives coding information from the control device 8 via the communication unit 25, which includes multiple images of a single item taken from multiple directions (step S31). The processor 21 stores the coding information, which includes the original image received from the control device 8 via the communication unit 25 and thumbnail images of the multiple captured images, in memory such as RAM23 or NVM24.

[0071] The processor 21 performs a second recognition process on the original image extracted from the coding information, recognizing the destination string using a second character recognition algorithm different from the character recognition performed by the control device 8 (step S32). If the processor 21 successfully recognizes the destination string through the second recognition process (step S33, YES), it sends destination information generated based on the recognized destination string to the control device 8 (step S34), and terminates the processing of the coding information (coding process).

[0072] If the processor 21 fails to recognize the destination string for the original image specified by the control device 8 (step S33, NO), it performs thumbnail processing as described in the first example of operation (steps S41-S44). As part of the thumbnail processing, the processor 21 first performs destination region detection processing on the thumbnail images of the captured images other than the original image (step S41). If the processor 21 cannot detect the destination region from the thumbnail images (step S42, NO), it records as the processing result of the thumbnail processing that the destination region could not be detected from images other than the original image, and proceeds to step S46.

[0073] If the processor 21 can detect the destination region from a captured image other than the original image (step S42, YES), it performs a replacement process to replace the captured image in which the destination region was detected with the original image (step S43). When the original image is replaced, the processor 21 recognizes the destination string on the replaced original image using a second character recognition process that is different from the character recognition process performed by the control device 8 (step S44). The processor 21 stores the recognition result of the destination string obtained from the recognition process on the replaced original image as the processing result of the thumbnail process.

[0074] Once the thumbnail processing is complete, the processor 21 determines whether or not it has successfully recognized the destination information as a result of the thumbnail processing (step S46). If the destination information has been successfully recognized by the thumbnail processing (step S46, YES), the processor 21 transmits the destination information recognized by the thumbnail processing to the control device 8 (step S34) and terminates the coding process for the coding information.

[0075] Furthermore, if the recognition of destination information by thumbnail processing fails (step S46, NO), the processor 21 displays the original image and thumbnail images of multiple captured images on the display unit 40 and performs key input processing to have the operator key in the destination information using the operation unit 30 (step S47). When the operator key in the destination information using the operation unit 30, the processor 21 transmits the destination information entered using the operation unit 30 to the control device 8 (step S34) and terminates the coding process for the coding information.

[0076] As described above, the VCD as a coding device according to the first embodiment acquires coding information including an original image selected by the control device of the item processing system from multiple captured images of the item to be processed taken from multiple directions, and thumbnail images of the multiple captured images. If the recognition of the destination string for the original image fails, the VCD detects the destination region from the thumbnail images, and performs a process to recognize the destination string by replacing the captured image of the thumbnail image in which the destination region was detected with the original image.

[0077] As a result, according to the first embodiment, even if character recognition fails on the original image selected by the item processing system, character recognition can be performed by replacing the original image with a thumbnail image in which the destination area was detected, thereby improving the final character recognition rate. As a result, the productivity of the destination information input work (coding process) by the coding device can be improved. (Second Embodiment) Next, a second embodiment will be described. Figure 9 shows an example configuration of a recognition system 102 including a coding device according to the second embodiment. The recognition system 102 according to the second embodiment includes an item processing system 1, a VCS2, and a second coding system 5 (5A, 5B, 5C, 5D). The item processing system 1 and VCS2 may have the same hardware configuration as those described in the first embodiment above. For this reason, a detailed explanation of the configuration of the item processing system 1 and VCS2 will be omitted.

[0078] However, in the second embodiment, each VCD20 constituting the VCS2 does not perform character recognition processing by the second algorithm, and the second coding system 5 is connected to the operation interface 26 and the display interface 27. The operation interface 26 of the VCD20 is connected to the keyboard (KB) / mouse emulator (also simply called emulator) 60 of the second coding system 5, which will be described later, and receives operation signals from the keyboard / mouse emulator 60. The display interface 27 of the VCD20 is connected to the capture board 70 of the second coding system 5, which will be described later, and transmits image data (display data for displaying the display screen on a display device) for displaying the display screen displayed by the VCD20 to the capture board 70.

[0079] In the second embodiment, when each VCD 20 receives coding information from the item processing system 1, it supplies a display screen showing the original image and thumbnail image included in the coding information to the second coding system 5 via the display interface 27, obtains destination information for the display screen from the second coding system 5 as key input information via the operation interface 26, and transmits the destination information based on the key input information obtained from the second coding system 5 to the item processing system 1.

[0080] Next, the configuration of a second coding system 5 having a second coding device 50, which is an example of a coding device according to the second embodiment, will be described. Furthermore, the second coding systems 5A to 5D have a similar configuration and will therefore be described as the second coding system 5, and the second coding devices 50a to 50d have a similar configuration and will therefore be described as the second coding device 5.

[0081] As shown in Figure 9, the second coding system 5 according to the second embodiment includes a second coding device 50, a keyboard / mouse emulator (emulator) 60, a capture board 70, an operation unit 80, and a display unit 90. The second coding device 50 and the VCD 20 are connected via the keyboard (KB) / mouse emulator 60 and the capture board 70. The operation unit 80 and the display unit 90 are connected to the second coding device 50.

[0082] Figure 10 is a block diagram showing an example configuration of a second coding device 50, which is an example of a coding device according to the second embodiment. As shown in Figure 10, the second coding device 50 includes a processor 51, ROM 52, RAM 53, NVM 54, communication unit 55, emulator interface 56, image interface 57, operation interface 58, and display interface 59, among others.

[0083] The processor 51 is connected to the ROM 52, RAM 53, NVM 54, communication unit 55, emulator interface 56, image interface 57, operation interface 58, and display interface 59 via a data bus or the like. The second coding device 50 may have configurations other than those shown in Figure 10 as needed, or certain configurations may be excluded from the configuration shown in Figure 10.

[0084] The processor 51 controls the operation of the entire second coding device 50. The processor 51 may include not only processing circuits for executing programs, but also an internal cache and various interfaces. The processor 51 performs various processes by executing programs pre-stored in the internal memory, ROM 52, or NVM 54. Furthermore, some of the various functions realized by the execution of a program by the processor 51 may be realized by hardware circuits. In this case, the processor 51 controls the functions executed by the hardware circuits.

[0085] ROM 52 is a non-volatile memory in which control programs and control data are pre-stored. The control programs and control data stored in ROM 52 are pre-loaded according to the specifications of the second coding device 50.

[0086] RAM 53 is volatile memory. RAM 53 temporarily stores data being processed by the processor 51. RAM 53 stores various application programs based on instructions from the processor 51. RAM 53 may also store data necessary for the execution of application programs and the execution results of application programs.

[0087] NVM54 is a non-volatile memory that allows data to be written to and rewritten. For example, NVM54 is composed of an HDD, SSD, or flash memory. NVM54 stores control programs, application programs, and various data according to the operational use of the second coding device 50.

[0088] The communication unit 55 is an interface for sending and receiving data. For example, the communication unit 55 may include an interface for sending and receiving data with another second coding device 50 or the like.

[0089] The emulator interface 56 connects to the keyboard / mouse emulator 60. The emulator interface 56 is an interface for sending and receiving data with the keyboard / mouse emulator 60. The emulator interface 56 causes the control signals from the processor 51 to be output to the VCD20 as operation signals via key input through the keyboard / mouse emulator 60. In other words, the emulator interface 56 inputs operation signals to the VCD20 through the keyboard / mouse emulator 60.

[0090] The image interface 57 is an interface for sending and receiving data with the capture board 70. The image interface 57 acquires the image of the display screen captured by the VCD20 on the capture board 70. The capture board 70 acquires the image of the display screen that the VCD20 displays on the display device (image data for display that the VCD20 supplies to the display device). The image interface 57 supplies the image of the VCD20 display screen captured by the capture board 70 to the processor 51.

[0091] The operation interface 58 is an interface for sending and receiving data with the operation unit 80. For example, the operation interface 58 receives an operation signal from the operation unit 80 indicating an operation input to the operation unit 80. The operation interface 58 transmits the received operation signal to the processor 51. The operation interface 58 may also supply power to the operation unit 80.

[0092] The display interface 59 is an interface for sending and receiving data with the display unit 90. The display interface 59 outputs image data from the processor 51 to the display unit 90.

[0093] The emulator interface 56, image interface 57, operation interface 58, and display interface 59 (or parts thereof) may be integrally formed on the main body of the second coding device 50. Furthermore, the second coding device 50 may include a device that performs a second recognition process and does not require operator key input. A second coding device 50 that does not require operator key input may not have an operation interface 58 and a display interface 59.

[0094] Next, we will briefly describe the processes performed by the second coding device 50, which is an example of a coding device according to the second embodiment. The processor 51 of the second coding device 50 realizes various processing functions by executing a program stored in the ROM 52 or NVM 54, etc.

[0095] First, the processor 51 acquires an image of the display screen from the VCD 20 through the image interface 57. As described in the first embodiment, the VCD 20 generates display (image) data for displaying a display screen that includes the original image and thumbnail images of multiple captured images included in the coding information from the control device 8. For example, as shown in Figure 5 or Figure 7, the VCD 20 generates display data for displaying a display screen (coding screen) on the display device that displays the original image in the main display area 41 and thumbnail images of multiple captured images in the sub-display area 42.

[0096] The capture board 70 acquires display data of the display screen (coding screen) from the VCD 20, captures an image of the display screen displayed by the display data, and supplies the captured image data of the display screen to the second coding device 50. In other words, the processor 51 of the second coding device 50 acquires the image data of the VCD 20 display screen captured by the capture board 70 via the image interface 57.

[0097] Furthermore, the processor 51 has the function of acquiring the display position, orientation, and size of the destination area of ​​the original image displayed in the main display area 41 on the display screen of the VCD 30. In addition, the processor 51 has the function of determining whether to input a transformation operation on the original image to the VCD 20 based on the position, orientation, and size of the destination area, and inputting an operation signal to the VCD 20 to instruct the transformation operation.

[0098] Here, the processor 51 detects the destination region in the original image and obtains the position, orientation, and size of the destination region. The processor 51 extracts the original image from the display screen of the VCD20 and detects the destination region in the extracted original image. The processor 51 obtains the position, orientation, and size of the destination region in the original image according to a predetermined algorithm. For example, the NVM 54 pre-stores a model (e.g., a neural network) that outputs the position, orientation, and size of the destination region. The processor 51 obtains the position, orientation, and size of the destination region in the original image based on the model stored in the NVM 54.

[0099] Furthermore, the NVM54 may be configured to pre-store a model that outputs the location of the destination region when a captured image is input, a model that outputs the orientation of the destination region when a captured image is input, and a model that outputs the size of the destination region when a captured image is input. In this case, the processor 51 inputs the original image extracted from the display screen into each model to obtain the location, orientation, and size of the destination region.

[0100] Furthermore, the processor 51 determines whether to input a transformation operation to the original image to the VCD20 based on the position, orientation, and size of the destination area. The processor 51 also determines whether it can appropriately perform character recognition processing on the destination area of ​​the original image displayed in the main display area 41 of the VCD20's display screen.

[0101] For example, the processor 51 determines whether the size of the destination area in the original image is greater than or equal to a predetermined size. If the size of the destination area is less than the predetermined size, the processor 51 determines that a transformation operation should be input to make the size of the destination area greater than or equal to the predetermined size. For example, if the size of the destination area is less than the predetermined size, the processor 51 determines that a transformation operation should be input to enlarge the original image in the main display area 41.

[0102] Furthermore, the processor 51 determines whether the orientation of the destination area in the original image is correct. If it determines that the orientation of the destination area is not correct, the processor 51 determines that a transformation operation should be input so that the orientation of the destination area in the original image is correct. For example, if the orientation of the destination area is tilted 90 degrees to the left, the processor 51 determines that a transformation operation should be input to rotate the original image in the main display area 41 90 degrees to the right.

[0103] Furthermore, the processor 51 determines whether the destination area of ​​the original image is cut off in the main display area 41 of the VCD20's display screen. If it determines that the destination area is cut off, the processor 51 determines to input a transformation operation so that the destination area of ​​the original image fits within the main display area 41. For example, if the right edge of the destination area is cut off, the processor 51 determines to input a transformation operation to move the original image in the main display area 41 to the left. Also, if the destination area of ​​the original image does not fit within the main display area 41, the processor 51 determines to input a transformation operation to shrink the original image in the main display area 41.

[0104] Furthermore, the processor 51 has a function to input an operation signal to the VCD20 instructing a transformation operation when it determines that a transformation operation on the original image is to be input to the VCD20. When it determines that a transformation operation is to be input, the processor 51 uses the keyboard / mouse emulator 26 to send an operation signal instructing the transformation operation that has been determined to be input to the operation interface 26 of the VCD20. The processor 51 causes the keyboard / mouse emulator 60 to generate an operation signal instructing the transformation operation via the emulator interface 56, and the keyboard / mouse emulator 60 outputs the operation signal instructing the transformation operation that it has generated to the operation interface 26 of the VCD20. As a result, the VCD20 outputs a display screen with the original image in the main display area 41 updated (transformed) according to the transformation operation via the display interface 27.

[0105] Furthermore, the processor 51 has a function to perform a replacement process to replace the original image displayed in the main display area 41 of the VCD30's display screen with another captured image. If the processor 51 fails to recognize the destination string by the second recognition process described later for the original image specified by the control device 8, it detects the destination area from the thumbnail image of a captured image other than the original image. If the destination area can be detected from the thumbnail image, the processor 51 inputs an operation signal to the VCD20 to instruct it to replace the original image displayed in the main display area 41 with the captured image from which the destination area was detected.

[0106] The processor 51 uses the keyboard / mouse emulator 60 to send an operation signal for a swap operation to the VCD20's operation interface 26 via the emulator interface 56, instructing the VCD20 to replace the original image. The processor 51 causes the keyboard / mouse emulator 60 connected via the emulator interface 56 to generate an operation signal instructing the swap operation, and the keyboard / mouse emulator 60 outputs the generated swap operation signal to the VCD20's operation interface 26. As a result, the VCD20 outputs a display screen via the display interface 27 with the original image in the main display area 41 replaced (updated) with the specified captured image in response to the swap operation.

[0107] Furthermore, the processor 51 has a function to recognize the destination string by a second recognition process for the destination area of ​​the original image. The processor 51 performs a character recognition process (second recognition process) to recognize the destination string for the destination area detected from the original image on the display screen of the VCD20, according to a predetermined algorithm (second algorithm) different from the first algorithm. The second algorithm is configured to recognize at least some of the characters that the first algorithm cannot recognize. The processor 51 identifies the destination information of the item based on the result of the second recognition process.

[0108] If the processor 51 can identify destination information through character recognition processing (second recognition processing), it inputs the identified destination information to the VCD 20. The processor 51 converts the destination information identified by the keyboard / mouse emulator 60 into an operation signal via key input and supplies it to the operation interface 26 of the VCD 20. In other words, the processor 51 generates an operation signal to input destination information to the keyboard / mouse emulator 4 via the emulator interface 56, and outputs the generated destination information to the operation interface 26 of the VCD 20 as an operation signal for key input.

[0109] Furthermore, if the second recognition process fails to recognize the destination string, the processor 51 performs a key input process in which the operator enters the destination information via the operation unit 80. If the second recognition process fails to recognize the destination string, the processor 51 displays the display screen from the VCD 20 on the display unit 90 and accepts the key input of destination information from the operation unit 80. The processor 51 inputs the destination information entered using the operation unit 80 to the VCD 20 as an operation signal indicating the key input information via the emulator 60 connected to the emulator interface 56.

[0110] Furthermore, the processor 51 acquires the display screen of the VCD captured by the capture board 70 in real time via the display interface 27 and displays the acquired display screen on the display unit 90. The operator visually confirms the original image displayed in the main display area 41 of the VCD 20 display screen on the display unit 90 and inputs destination information using the operation unit 80. Once the destination input is complete, the operator inputs an operation to the operation unit 80 to complete the input of destination information.

[0111] The second coding system 5 may also include a system in which the operation unit 80 and the display unit 90 are not connected to the second coding device 50. In this case, the processor 51 of the second coding device 50, which is not connected to the operation unit 80 and the display unit 90, may display the display screen of the VCD 20 on the display unit 90 connected to the other second coding device 50, and input operation signals to the VCD 20 indicating the operation input to the operation unit 80 connected to the other second coding device 50. Furthermore, the main second coding device 50 (for example, the second coding device 50a) or an external control device may manage the operation unit 80 used for destination input and the display unit 90 that displays the display screen of the VCD 20.

[0112] Next, the operation of the second coding device 50 in the second coding system 5 of the recognition system 102 according to the second embodiment will be described. Figure 11 is a flowchart illustrating a first example of operation of the second coding device 50 as a coding device according to the second embodiment.

[0113] The second coding device 50 acquires the display screen output by the VCD 20 as image data of the display screen captured by the capture board 70 via the image interface 57 (step S51). The processor 21 of the second coding device 50 stores the image data of the display screen acquired via the image interface 57 in memory such as RAM 53 or NVM 54. Here, we will explain assuming that image data of the display screen of the first display example shown in Figure 5 has been acquired.

[0114] The processor 51 extracts the original image to be displayed in the main display area 41 of the display screen from the image data of the display screen of the VCD 20. The processor 51 performs a second recognition process on the extracted original image to recognize the destination string using a second algorithm for character recognition that is different from the character recognition performed by the control device 8 (step S52).

[0115] If the processor 51 successfully recognizes the destination string through the second recognition process (step S53, YES), it outputs destination information based on the destination string obtained through the second recognition process to the emulator 60 via the emulator interface 56 (step S54). The emulator 60 generates an operation signal to input the destination information output from the emulator interface 56 as a key input and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8 and terminates the coding process for the coding information.

[0116] Furthermore, in parallel with the second recognition process for the original image, the processor 51 performs a thumbnail process to recognize destination information from captured images other than the original image included in the coding information (steps S61-S64).

[0117] The processor 51, as part of the thumbnail processing, first performs a detection process to detect the destination region in the thumbnail images of captured images other than the original image (step S61). If the processor 51 cannot detect the destination region from any of the thumbnail images of captured images (step S62, NO), it stores a note indicating that the destination region could not be detected from the thumbnail image as the processing result of the thumbnail processing.

[0118] Furthermore, if the processor 51 can detect the destination region from the thumbnail images of the captured images other than the original image (step S62, YES), it performs a replacement operation to replace the captured image of the thumbnail image in which the destination region was detected with the original image. The processor 51 sends a replacement operation to the keyboard / mouse emulator 60 via the emulator interface 56 to replace the captured image of the thumbnail image in which the destination region was detected with the original image (step S63).

[0119] The keyboard / mouse emulator 60 generates an operation signal to instruct a swap operation and sends the generated swap operation signal to the VCD20's operation interface 26. As a result, when the VCD20's processor 21 receives the swap operation signal via the operation interface 26, it generates (updates) a display screen in which the original image in the main display area 41 has been replaced with the specified captured image according to the operation signal, and outputs the image data of the generated display screen via the display interface 27.

[0120] The capture board 70 captures the display screen with the original image output by the display interface 27 replaced, and transmits the image data of the captured display screen to the image interface 57 of the second coding device 50. The processor 51 of the second coding device 50 acquires the display screen with the original image replaced, which was received by the image interface 57 (step S64).

[0121] For example, in the display screen of the first display example shown in Figure 5, the captured image displayed in the fifth area 425 of the sub-display area 42 is displayed as the original image in the main display area 41. In the display screen of the first display example shown in Figure 5, if a destination area is detected in the thumbnail image displayed in the first display area 421 of the sub-display area 42, the processor 51 sends an operation signal to the keyboard / mouse emulator 60 via the emulator interface 56 to replace the captured image in the thumbnail image displayed in the first display area 421 with the original image. The keyboard / mouse emulator 60 generates an operation signal instructing the capture image in the thumbnail image of the first display area 421 to be replaced with the original image, and sends the generated operation signal to the operation interface 26 of the VCD 20.

[0122] The VCD20 processor 21 receives an operation signal via the operation interface 26 instructing it to replace the captured image in the first display area 421 with the original image. The VCD20 processor 21 and the control device 8 acquire the full-size captured image of the thumbnail image (the thumbnail image to be replaced) displayed in the first display area 421.

[0123] Alternatively, the full-size image data of multiple captured images displayed as thumbnail images in the sub-display area 42 may be received as coding information from the control device 8 and stored in memory such as RAM 23. In this case, the processor 21 of the VCD 20 may acquire the full-size captured images of the thumbnail images displayed in the first display area 421 from memory.

[0124] When the VCD20's processor 21 acquires a full-size captured image of a thumbnail image that has been specified for replacement, it generates (updates) a display screen as shown in Figure 7 by displaying the acquired full-size captured image as the original image in the main display area 41. When the VCD's processor 21 generates (updates) a display screen in which the original image in the main display area 41 has been replaced with the captured image in the first display area 421, it outputs the display data of the generated display screen via the display interface 27.

[0125] The capture board 70 captures an image of the display screen in which the original image output by the VCD 20 has been replaced with the image of the first display area 421, and transmits the captured image data of the display screen to the image interface 57 of the second coding device 50. As a result, the processor 51 of the second coding device 50 acquires a display screen as shown in Figure 7, in which the original image received by the image interface 57 has been replaced.

[0126] When the processor 21 obtains a display screen with the original image replaced, it performs an enlargement and rotation (deformation) operation on the original image on the display screen obtained through the replacement process (step S65). As described above, the processor 21 detects the destination region in the original image and determines whether to input a deformation operation on the original image to the VCD20 based on the position, orientation, and size of the destination region.

[0127] If the processor 51 determines that no deformation operation is input (i.e., no deformation operation is required), it proceeds to step S66. If it determines that a deformation operation is input, the processor 51 generates deformation instructions that instruct the original image to move, enlarge, rotate, etc., and supplies deformation operation signals to the VCD20 via the emulator interface 56 and emulator 60.

[0128] The VCD20 generates a display screen that displays the deformed (updated) original image in the main display area 41 in response to the deformation operation signal input to the operation interface 26, and outputs it from the display interface 27. The display screen showing the deformed original image is captured by the capture board 70, and the image data of the captured display screen is supplied to the image interface 57 of the second coding device 50. As a result, the processor 51 of the second coding device 50 acquires the display screen showing the deformed original image in response to the deformation operation.

[0129] When the processor 51 acquires a display screen in which the original image has been replaced or a display screen in which the replaced original image has been transformed, it recognizes the destination string on the original image of the acquired display screen using a second algorithm for character recognition, which is different from the character recognition of the control device 8 (step S66). The processor 51 stores the recognition result of the destination string by the character recognition of the replaced original image using the second algorithm as the processing result of thumbnail processing.

[0130] If the processor 51 cannot recognize the destination string from the replaced original image, it stores a note in the thumbnail processing result indicating that the destination information could not be recognized from the captured image of the thumbnail. If the processor 51 can recognize the destination string from the replaced original image, it stores destination information, including the recognized destination string, in the thumbnail processing result.

[0131] On the other hand, if the processor 51 fails to recognize the destination string for the original image specified by the control device 8 (step S53, NO), it determines whether or not the thumbnail processing described above has been completed (step S55). If the thumbnail processing has not been completed (step S55, NO), the processor 51 waits for the thumbnail processing to be completed.

[0132] If thumbnail processing is completed (step S55, YES), the processor 51 determines whether or not it has successfully recognized the destination information as a result of the thumbnail processing (step S56). If the destination information has been successfully recognized by thumbnail processing (step S56, YES), the processor 51 supplies the destination information recognized by thumbnail processing to the emulator 60 via the emulator interface 56 (step S54). The emulator 60 generates an operation signal to input the destination information received from the emulator interface 56 of the second coding device 50 as a key input, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and terminates the coding process for the coding information.

[0133] Furthermore, if the recognition of destination information by thumbnail processing fails (step S56, NO), the processor 51 displays a display screen as shown in Figure 5 or Figure 7, acquired by the image interface 57, on the display unit 90, and performs key input processing to have the operator input destination information using the operation unit 80 (step S57).

[0134] The operator inputs destination information using the operation unit 80 while visually observing (referencing) the display screen shown in Figure 5 or Figure 7 displayed on the display unit 90. The processor 51 of the second coding device 50 acquires the destination information that the operator inputs using the operation unit 80. When the processor 21 acquires the destination information that the operator inputs using the operation unit 80, it supplies the acquired destination information to the emulator 60 via the emulator interface 56 (step S54). The emulator 60 generates an operation signal to input the destination information received from the emulator interface 56 of the second coding device 50 and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8 and completes the coding process for the coding information.

[0135] Next, a second example of operation of the second coding device 50 as a coding device according to the second embodiment will be described. In the second example of operation, the thumbnail processing that was performed as parallel processing in the first example of operation is performed when the recognition of the destination string for the original image specified by the control device 8 fails. Figure 12 is a flowchart illustrating a second example of operation of the second coding device 50 as a coding device according to the second embodiment.

[0136] The second coding device 50 acquires the display screen output by the VCD 20 as image data of the display screen captured by the capture board 70 via the image interface 57 (step S71). The processor 21 of the second coding device 50 stores the image data of the display screen acquired via the image interface 57 in memory such as RAM 53 or NVM 54. Here, we will explain assuming that image data of the display screen of the first display example shown in Figure 5 has been acquired.

[0137] The processor 51 extracts the original image to be displayed in the main display area 41 of the display screen from the image data of the display screen of the VCD 20. The processor 51 performs a second recognition process on the extracted original image to recognize the destination string using a second algorithm for character recognition that is different from the character recognition performed by the control device 8 (step S72).

[0138] If the processor 51 successfully recognizes the destination string in the second recognition process (step S73, YES), it outputs destination information based on the destination string obtained in the second recognition process to the emulator 60 via the emulator interface 56 (step S74). The emulator 60 generates an operation signal to input the destination information received from the emulator interface 56 as a key input and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8 and completes the coding process for the coding information.

[0139] Furthermore, if the processor 51 fails to recognize the destination string by the second recognition process on the original image specified by the control device 8 (step S73, NO), it performs a thumbnail process to recognize the destination information from captured images other than the original image that are displayed as thumbnail images (steps S81-S86).

[0140] The processor 51 performs a detection process to detect the destination region in the thumbnail images of captured images other than the original image as part of the thumbnail processing (step S81). If the destination region cannot be detected from any of the thumbnail images of captured images (step S82, NO), the processor 51 stores a note indicating that the destination region could not be detected from the thumbnail image as the processing result of the thumbnail processing.

[0141] Furthermore, if the processor 51 can detect the destination area from the thumbnail images of captured images other than the original image (step S82, YES), it sends a swap operation to the keyboard / mouse emulator 60 via the emulator interface 56 to replace the captured image of the thumbnail image in which the destination area was detected with the original image (step S83). As a result, the keyboard / mouse emulator 60 sends an operation signal instructing the swap operation to the operation interface 26 of the VCD20. The processor 21 of the VCD20 generates (updates) a display screen in which the original image of the main display area 41 has been replaced with the specified captured image in response to the swap operation signal received via the operation interface 26, and outputs the image data of the generated display screen via the display interface 27.

[0142] The capture board 70 captures the display screen with the original image output by the display interface 27 replaced, and transmits the image data of the captured display screen to the image interface 57 of the second coding device 50. As a result, the processor 51 of the second coding device 50 acquires the display screen with the original image replaced, which was received by the image interface 57 (step S84).

[0143] Furthermore, when the processor 21 obtains a display screen with the original image replaced, it performs an enlargement and rotation operation (deformation operation) on the display screen obtained by the replacement process (step S85). If the processor 51 determines that no deformation operation is input (that no deformation operation is necessary), it proceeds to step S66. If it determines that a deformation operation is input, the processor 51 generates a deformation instruction that instructs the original image to move, enlarge, rotate, etc., and supplies the deformation operation signal to the VCD20 via the emulator interface 56 and emulator 60.

[0144] The VCD20 generates a display screen that displays the deformed (updated) original image in the main display area 41 in response to the deformation operation signal input to the operation interface 26, and outputs it from the display interface 27. The display screen showing the deformed original image is captured by the capture board 70, and the image data of the captured display screen is supplied to the image interface 57 of the second coding device 50. As a result, the processor 51 of the second coding device 50 acquires the display screen showing the deformed original image in response to the deformation operation.

[0145] When the processor 51 acquires a display screen in which the original image has been replaced or a display screen in which the replaced original image has been transformed, it recognizes the destination string on the original image of the acquired display screen using a second algorithm for character recognition, which is different from the character recognition of the control device 8 (step S86). The processor 51 stores the recognition result of the destination string by the character recognition of the replaced original image using the second algorithm as the processing result of thumbnail processing.

[0146] If the processor 51 cannot recognize the destination string from the replaced original image, it stores a note in the thumbnail processing result indicating that the destination information could not be recognized from the captured image of the thumbnail. If the processor 51 can recognize the destination string from the replaced original image, it stores destination information, including the recognized destination string, in the thumbnail processing result.

[0147] The processor 51 determines whether it has successfully recognized the destination information as a result of the thumbnail processing (step S87). If the destination information is successfully recognized by the thumbnail processing (step S87, YES), the processor 51 supplies the destination information recognized by the thumbnail processing to the emulator 60 via the emulator interface 56 (step S74). The emulator 60 generates an operation signal to input the destination information received from the emulator interface 56 of the second coding device 50 as a key input, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and terminates the coding process for the coding information.

[0148] Furthermore, if the recognition of destination information by thumbnail processing fails (step S87, NO), the processor 51 displays a display screen as shown in Figure 5 or Figure 7, acquired by the image interface 57, on the display unit 90, and performs key input processing to have the operator input destination information using the operation unit 80 (step S89).

[0149] When the processor 51 acquires destination information entered by the operator using the operation unit 80, it supplies the acquired destination information to the emulator 60 via the emulator interface 56 (step S74). The emulator 60 generates an operation signal to input the destination information received from the emulator interface 56 of the second coding device 50, and supplies the generated operation signal to the VCD 20. As a result, the VCD 20 transmits the destination information indicated by the operation signal input to the operation interface 26 to the control device 8, and completes the coding process for the coding information.

[0150] As described above, the second coding device, as a coding device according to the second embodiment, is connected to the display interface and operation interface of the VCD in the existing VCS. The second coding device acquires an image of the display screen of the existing VCD, which includes an original image selected from multiple captured images of the item to be processed and thumbnail images of the multiple captured images. If the second coding device fails to recognize the destination string for the original image extracted from the display screen acquired from the existing VCD, it supplies an operation signal to the existing VCD for a replacement operation to replace the captured image of the thumbnail image in which the destination area could be detected with the original image. The second coding device acquires the display screen with the replaced original image from the existing VCD and performs processing to recognize the destination string for the replaced original image.

[0151] This allows the system to perform character recognition on the original image in the display screen acquired from the VCD, even if character recognition fails. If the destination area can be detected from the thumbnail image of the display screen, the system can replace the original image with the image of the thumbnail in which the destination area was detected and perform character recognition again, thereby improving the final character recognition rate. As a result, the recognition rate of destination information by the second coding device can be improved without changing the configuration of the existing VCD, thereby improving the productivity of the coding process.

[0152] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of symbols]

[0153] 1...Item processing system, 2...VCS, 5 (5A, 5B, 5C, 5D)...Second coding system, 6...Sorter, 7...Camera, 8...Control device, 11...Processor, 12...ROM, 13...RAM, 14...NVM, 15...Camera interface, 16...Communication unit, 19...Sorter interface, 20 (20a, 20b, 20c, 20d)...VCD (coding device), 21...Processor, 22...ROM, 23...RAM, 24...NVM, 25...Communication unit, 26...Operation interface, 27...Display interface, 30...Operation unit, 40...Display unit, 50 (50 a, 50b, 50c, 50d)...Second coding device (coding device), 51...Processor, 52...ROM, 53...RAM, 54...NVM, 55...Communication unit, 56...Emulator interface, 57...Image interface, 58...Operation interface, 59...Display interface, 60 (60a, 60b, 60c, 60d)...Keyboard (KB) / mouse emulator, 70 (70a, 70b, 70c, 70d)...Capture board, 80 (80a, 80b, 80c, 80d)...Operation unit, 90 (90a, 90b, 90c, 90d)...Display unit.

Claims

1. A communication unit that communicates with an item processing system that processes items, and receives from the item processing system a first original image selected by the control device of the item processing system from multiple images of the item taken from multiple directions, and thumbnail images of the images other than the first original image. A processor that performs the following: recognition processing to recognize the destination string for the first original image; detecting the destination region from thumbnail images of captured images other than the first original image, acquiring the captured image corresponding to the thumbnail image in which the destination region was detected as the second original image, and performing thumbnail processing to recognize the destination string for the second original image; A coding device having

2. The first original image received by the communication unit from the item processing system is an image of the item in which the control device of the item processing system failed to recognize the destination string in character recognition processing using the first algorithm. The recognition process performed by the processor recognizes the destination string by character recognition processing according to a second algorithm different from the first algorithm. The coding apparatus according to claim 1.

3. An operating interface connected to the control unit, It includes a display interface that connects to the display unit, If the processor fails to recognize the destination string for the first original image received from the item processing system by the recognition process, and fails to recognize the destination string for the second original image by the thumbnail processing, it displays the first or second original image received from the item processing system on the display unit and transmits the destination information entered on the operation unit to the item processing system. The coding apparatus according to claim 2.

4. An image interface for acquiring image data that captures a first display screen including a first original image selected from multiple captured images taken from multiple directions of an item displayed on a display device by another coding device, and thumbnail images of captured images other than the first original image. An emulator interface connected to an emulator that generates operation signals to be input to the other coding device, A processor that performs recognition processing to recognize the destination string for the first original image on the first display screen acquired as image data by the image interface, and after sending a swapping operation to the emulator via the emulator interface to instruct the generation of a second display screen in which the captured image corresponding to the thumbnail image in which the destination region was detected is displayed as the target of recognition of the destination string, the image interface acquires image data of the second display screen that displays the second original image generated by the other coding device in response to the swapping operation signal from the emulator, and performs thumbnail processing to recognize the destination string for the second original image on the second display screen. A coding device having

5. The aforementioned processor, The emulator interface transmits a transformation operation to the emulator that transforms the second original image according to the state of the destination region in the second original image of the second display screen. The image interface acquires image data of the transformed display screen that displays the transformed second original image generated by the other coding device in response to the transformation operation signal from the emulator. The string of characters for the destination to the transformed second original image on the transformed display screen is recognized by the recognition process. The coding apparatus according to claim 4.

6. The image interface is connected to the display interface of another coding device connected to the control device of the article processing system that processes the article, The emulator interface is connected to the operating interface of the other coding device via the emulator. The processor recognizes the first original image and the destination string for the second original image by performing character recognition processing according to a second algorithm different from the first algorithm used for character recognition processing performed by the control device of the item processing system. A coding apparatus according to any one of claims 4 or 5.

7. An operating interface connected to the control unit, It includes a display interface that connects to the display unit, If the processor fails to recognize the destination string for the first original image by the recognition process and to recognize the destination string for the second original image by the thumbnail processing, it displays the first display screen or the second display screen on the display unit and transmits the destination information input to the operation unit to the emulator via the emulator interface. The coding apparatus according to claim 6.

8. A computer having a communication unit that communicates with an article processing system that processes articles and a processor that executes a program, A coding processing program that causes the coding device described in claim 1 to perform processing.

9. A computer comprising: an image interface that acquires an image of a coding screen captured from multiple images taken from multiple directions of an item displayed on a display device by another coding device, including an original image selected from multiple images and thumbnail images of images other than the original image; an emulator interface that connects to an emulator that generates operation signals to be input to the other coding device that acquired the image via the image interface; and a processor that executes a program. A coding processing program that causes the coding device described in claim 4 to perform processing.

Images