Optical information reading device

Abstract

The present invention provides an optical information reading device capable of reading structured data from an object to which symbols are attached as a matrix. [Solution] The optical information reading device 10 includes an Eima light irradiation unit 32 that irradiates Eima light 40 extending in a first direction 40H. Two symbols from the object 11 are designated as first symbols and second symbols, and a first distance is calculated between the second symbol and a first straight line that passes through a first reference point defined for the first symbol and is parallel to the first direction 40H, and a second distance is calculated between the second symbol and a second straight line that passes through a second reference point defined for the first symbol and intersects the first direction 40H. Based on the first and second distances, the row and column in a matrix are identified for each of the multiple symbols 20, and the symbol information represented by the symbols 20 is output as structured data.

Description

Technical Field

[0001] The present disclosure relates to an optical information reading device.

Background Art

[0002] Patent Document 1 discloses a table processing method and a table processing device. In the table processing method and the table processing device described in Patent Document 1, grid lines are extracted from an image, and further, an inner table frame surrounded by the grid lines is extracted. From the inner table frame, item frames and data frames are extracted, and the item names and the entered contents described in each of the extracted item frames and data frames are subjected to character recognition.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the table processing method and the table processing device described in Patent Document 1, grid lines are extracted from an image, but the process of extracting grid lines from an image increases the amount of calculation, so a processor with high computing power is required. Further, in the table processing method and the table processing device described in Patent Document 1, if the image does not include grid lines, the inner table frame cannot be extracted.

[0005] In view of the above problems, an object of the present disclosure is to provide an optical information reading device capable of reading structured data from an object to which symbols are attached as a matrix.

Means for Solving the Problems

[0006] According to one aspect of the present disclosure, an optical information reading device is an optical information reading device that images an object to which a plurality of symbols are attached as a matrix in which a plurality of symbols are aligned in a row direction and in a column direction intersecting the row direction, and reads symbol information represented by the plurality of symbols, comprising: an AIM light irradiation unit that irradiates AIM light extending in a first direction intersecting the irradiation direction toward the object in the direction of irradiation; an imaging unit that has an imaging field toward the irradiation direction and generates an input image that includes the symbols attached to the object by imaging a region within the imaging field; a matrix processing unit that performs matrix processing based on the input image; and an output unit that outputs the symbol information represented by the plurality of symbols attached to the object as structured data of the structured matrix based on the result of the matrix processing by the matrix processing unit, wherein the matrix processing unit detects the plurality of symbols included in the input image by performing the matrix processing on the input image, and the plurality Two of the symbols are designated as the first symbol and the second symbol, and a first distance is calculated between the second symbol and a first straight line that passes through a first reference point defined for the first symbol and is parallel to the first direction, and a second distance is calculated between the second symbol and a second straight line that passes through a second reference point defined for the first symbol and intersects the first direction, and based on the first and second distances between the first and second symbols, it is determined whether the first symbol and the second symbol are in the same row in the matrix and whether the first symbol and the second symbol are in the same column in the matrix, thereby identifying the row and column in the matrix for each of the multiple symbols, and the output unit outputs the symbol information represented by the multiple symbols as the structured data of the matrix based on the row and column identified by the matrix processing unit. [Effects of the Invention]

[0007] According to an optical information reading device in one aspect of the present disclosure, it is possible to read structured data from an object to which symbols are attached as a matrix. [Brief explanation of the drawing]

[0008] [Figure 1] A schematic diagram showing the external appearance of an optical information reading device. [Figure 2] A block diagram schematically showing the configuration of an information processing system including an optical information reading device. [Figure 3] A block diagram schematically showing the configuration of the imaging module. [Figure 4] A block diagram schematically showing the configuration of the control unit. [Figure 5] A block diagram schematically showing the configuration of the memory unit. [Figure 6] A flowchart illustrating the overall operation of an optical information reading device. [Figure 7] A flowchart showing the order of character detection, character concatenation, and string filtering. [Figure 8] This diagram illustrates an example of a procedure for narrowing down the output string from multiple characters contained in the input image. [Figure 9] A diagram explaining string margins. [Figure 10] A diagram showing the format registration screen. [Figure 11] A diagram showing an example of an object with multiple strings attached. [Figure 12] This diagram illustrates how output strings are filtered based on string likelihood. [Figure 13] This diagram illustrates how output strings are filtered based on their degree of match with a format pattern. [Figure 14] This diagram illustrates how output strings are filtered based on the distance to the targeting position. [Figure 15] A schematic diagram showing the correspondence between the AI ​​processing core and the image processing core included in the Information Processing Unit. [Figure 16] A diagram showing how the AI ​​processing core and the image processing core operate in parallel. [Figure 17] A flowchart illustrating the string processing flow using the first and second models. [Figure 18] A flowchart illustrating a portion of the string processing shown in Figure 17. [Figure 19] The figure explaining the string narrowing based on the extending direction of the string candidate. [Figure 20] The figure showing an example of the first type of object with a plurality of symbols attached. [Figure 21] The figure showing an example of the second type of object with a plurality of symbols attached. [Figure 22] The flowchart showing the flow of the reading process including the relative reading process. [Figure 23] The flowchart showing the flow of the reading target setting for the relative reading process. [Figure 24] The figure showing the reading target setting screen at the start of the reading target setting for the first type of object. [Figure 25] The figure showing the reading target setting screen in a state of receiving the user-specified position for the first type of object. [Figure 26] The figure showing the reading target setting screen in a state of displaying the confirmation message for the specified position setting. [Figure 27] The figure showing the setting state of the reading target for the first type of object. [Figure 28] The figure showing the confirmation message for the specified position setting regarding the second type of object. [Figure 29] The figure showing the setting state of the reading target for the first type of object and the second type of object. [Figure 30] The figure showing the reading target setting screen in a state of displaying the message prompting to start the user application. [Figure 31] The figure showing the application selection screen. [Figure 32] The figure showing the screen of the user application having a plurality of input fields. [Figure 33] The figure showing the selection screen of the user-specified position corresponding to the input field. [Figure 34] The figure showing the correspondence between the symbol on the object and the input field of the user application. ' [Figure 35] The figure showing an example of an object with a plurality of symbols attached as a matrix. [Figure 36] The flowchart showing the flow of the matrix processing. [Figure 37]This figure shows an example of an input image obtained when the object is captured while tilted. [Figure 38] A diagram illustrating the first distance between each symbol. [Figure 39] A diagram illustrating the second distance between each symbol. [Figure 40] This diagram shows the case where the first and second reference points are the same. [Figure 41] This figure shows an example of a pre-configuration screen for matrix processing. [Figure 42] A diagram showing an example of a format pattern and input image. [Figure 43] A diagram illustrating an example of how structured data can be used as master data. [Figure 44] A flowchart illustrating the process of excluding information that does not match the format information. [Figure 45] This diagram illustrates an example of when information that does not match the formatting information is excluded from structured data. [Figure 46] A diagram illustrating matrix processing that takes perspective into account. [Figure 47] A diagram illustrating matrix processing that takes into account the angles between symbols. [Modes for carrying out the invention]

[0009] Embodiments of this disclosure will be described below with reference to the drawings. In the drawings, the same or corresponding parts will be denoted by the same reference numerals to avoid repetition in the description. In the following description, terms such as "up," "down," "left," and "right" may be used to indicate position or direction. These terms are used for convenience to facilitate understanding of the embodiments and are not related to the actual direction in which they are implemented unless otherwise explicitly stated.

[0010] Hereinafter, an optical information reading device 10 according to an example of an embodiment of the present disclosure will be described with reference to the drawings. First, the configuration of the optical information reading device 10 will be described with reference to Figure 1. Figure 1 is a schematic diagram showing the external appearance of the optical information reading device 10.

[0011] The optical information reading device 10 images an object 11 to be read, which has a symbol 20 attached to it, and reads the symbol information represented by the symbol 20. The symbol 20 represents information in an optically readable form, and for example, standardized codes such as barcodes and two-dimensional codes, or proprietary codes can be used as symbols 20. The symbol 20 only needs to be optically readable, and may be a string of characters that directly represents the content of the information using letters and numbers. The symbol 20 is displayed on the surface of the item being worked on. For example, the symbol 20 may be printed directly on the surface of a commercial product, or a label with the symbol 20 printed on it may be attached to the surface of the item.

[0012] The optical information reading device 10 reads the symbol information represented by the symbol 20 by decoding (decoding) the symbol information encoded in a code such as a barcode or a two-dimensional code, or by performing OCR (optical character recognition) processing on the string.

[0013] The optical information reading device 10 is a device that can be held and carried by an operator (not shown) who performs work using the symbol information represented by the symbol 20, and is sometimes called a handheld terminal. The optical information reading device 10 comprises an imaging module 13, a display unit 14, and an operation unit 15. The optical information reading device 10 is covered by a housing 17, and the display unit 14 and the operation unit 15 are located on the top surface of the housing 17. In addition to a handheld terminal as a dedicated device for work using the symbol information represented by the symbol 20, a smartphone with application software installed for performing work using the symbol information may also be used as a smartphone-type optical information reading device 10.

[0014] The display unit 14 displays various information to the operator. The display unit 14 is, for example, a liquid crystal display (LCD) or an organic EL display. The operation unit 15 receives various inputs to the optical information reading device 10. The operation unit 15 includes a trigger key 18 that the operator operates when making the imaging module 13 perform imaging. In addition to the trigger key 18, the operation unit 15 includes multiple operation keys such as a numeric keypad, power key, and function key. The display unit 14 may also be a touch panel display that also functions as the operation unit 15.

[0015] The housing 17 in Figure 1 is formed to be elongated in one direction (the longitudinal direction). The display unit 14 is located on the upper surface of one end of the housing 17 in the longitudinal direction. The operating unit 15 is located on the upper surface of the other end of the housing 17. The other end of the housing 17 is narrower in width than the end with the display unit 14, making it easier for the operator to grasp the other end where the operating unit 15 is located.

[0016] The imaging module 13 is located on the front of the tip of one end of the housing 17, which is pointed towards the object 11. The operator holds the other end of the housing 17 and carries the optical information reader 10. When the operator images the object 11 with the optical information reader 10, they point the tip on which the imaging module 13 is located towards the object 11.

[0017] The imaging module 13 is a module that includes an imaging unit 31 that captures an area within the imaging field of view and generates an image such as an input image. The imaging unit 31 is a camera unit that includes, for example, an image sensor such as a CMOS or CCD that converts light into electrical signals, and an imaging optical system (such as a lens) that collects the reflected light from the symbol 20 and causes it to enter the image sensor. The imaging unit 31 in Figure 1 has multiple (in this case, three) camera units. Each of these camera units has a different focal length, making it possible for the imaging unit 31 as a whole to capture objects 11 at various distances with high image quality. The imaging unit 31 may include, for example, a short-range camera unit with a short fixed focal length, a long-range camera unit with a long fixed focal length, and a variable intermediate camera unit whose focal length is variable between short and long distances. The image generated by the imaging unit 31 is an input image that includes, for example, a symbol (such as a string of characters) attached to the object 11 and is used for reading symbol information by the optical information reading device 10. In addition, the imaging unit 31 may also generate images used for purposes such as operational images for executing reading processing by the optical information reading device 10, and configuration images for configuring the optical information reading device 10.

[0018] The imaging module 13 in Figure 1 includes, in addition to the imaging unit 31, an AIM light irradiation unit 32 and an illumination unit 33. The AIM light irradiation unit 32 irradiates AIM light 40 toward the object 11 in irradiation direction A. The illumination unit 33 irradiates illumination light to assist the imaging unit 31 in capturing the symbol 20. The imaging module 13 in Figure 1 is equipped with multiple (two in this case) light sources as the illumination unit 33. By irradiating illumination light from multiple directions, shadows on the object 11 are reduced, and the accuracy of reading the symbol 20 is improved.

[0019] The irradiation direction A toward the object 11 is the direction from the front of the tip of the housing 17, which is pointed toward the object 11, toward the object 11. The imaging unit 31 has an imaging field of view toward this irradiation direction A and images the area within the imaging field of view. The AIM light 40 determines the aiming position of the optical information reading device 10. In the information processing performed by the optical information reading device 10, the aiming position is treated as the position where the operator is aiming toward the object 11. For example, if the object 11 has multiple symbols 20 attached to it, the symbol 20 closest to the AIM light 40 may be treated as the target for reading.

[0020] In Figure 1, the AIM light 40 includes a first AIM light 40a, which is a linear beam of light extending in the first direction 40H, and a second AIM light 40b, which is a linear beam of light intersecting the first direction 40H. In Figure 1, the second AIM light 40b, which is shorter than the first AIM light 40a, intersects the center of the first AIM light 40a. In Figure 1, the second AIM light 40b extends in the second direction 40V, which is perpendicular to the first AIM light 40a. However, depending on the orientation of the optical information reading device 10, the second AIM light 40b may intersect the first AIM light 40a at an angle that is not perpendicular. Also, the AIM light 40 only needs to include the first AIM light 40a extending in the first direction 40H, and the second AIM light 40b is not necessarily required.

[0021] The EIM light irradiation unit 32 in Figure 1 includes a first irradiation unit 32a that irradiates first EIM light 40a extending in a first direction 40H, and a second irradiation unit 32b that irradiates second EIM light 40b extending in a direction intersecting the first direction 40H. The first irradiation unit 32a and the second irradiation unit 32b are each units that include light-emitting elements such as LEDs (light-emitting diodes).

[0022] The first direction 40H is a direction that intersects with the irradiation direction A, and the first Eima light 40a extends within the plane in which the symbol 20 is displayed on the object 11. The second direction 40V is a direction that intersects with both the irradiation direction A and the first direction 40H, and in Figure 1, it is a direction perpendicular to the first direction 40H within the plane in which the symbol 20 is displayed.

[0023] The orientation of the first direction 40H changes according to the inclination of the optical information reading device 10 and the object 11. For example, if the operator is pointing the optical information reading device 10 horizontally at an object 11 that extends vertically, the first direction 40H will be the horizontal direction within the plane of the object 11.

[0024] The first direction 40H of the first AIM light 40a of the AIM light 40 may be used as a reference to determine the reading direction of the symbol 20 when the optical information reading device 10 reads the symbol 20. Therefore, it is preferable for the operator to irradiate the object 11 with the AIM light 40a so that the first AIM light 40a is aligned with the direction of alignment of the symbols 20. For the purposes of the explanation below, it will be assumed that the first AIM light 40a is irradiated so as to be aligned with the direction of alignment of the symbols 20. Furthermore, unless otherwise specified, the aiming position determined by the AIM light 40 will be the position where the first AIM light 40a and the second AIM light 40b intersect on the object 11 (the center of the first AIM light 40a).

[0025] In addition, the optical information reading device 10 may include a power supply unit (not shown). The power supply unit supplies power to drive the optical information reading device 10. For example, a battery unit that can be attached to and detached from the housing 17 can be used as the power supply unit.

[0026] Next, the configuration of the information processing system 100, including the optical information reading device 10, will be described with reference to the block diagrams in Figures 2, 3, 4, and 5. Figure 2 is a schematic block diagram showing the configuration of the information processing system 100, including the optical information reading device 10. Figure 3 is a schematic block diagram showing the configuration of the imaging module 13. Figure 4 is a schematic block diagram showing the configuration of the control unit 50. Figure 5 is a schematic block diagram showing the configuration of the storage unit 60.

[0027] The optical information reading device 10 includes an imaging module 13, a display unit 14, and an operation unit 15, as well as an output unit 12, a control unit 50, and a storage unit 60. The display unit 14 is part of the output unit 12.

[0028] The output unit 12 is a unit that performs signal processing to output data processed within the optical information reading device 10 to the outside of the optical information reading device 10. In addition to the display unit 14, the output unit 12 includes a communication unit 16, and the optical information reading device 10 communicates with external devices, such as a host computer 19, via the communication unit 16. The communication unit 16 is a communication interface unit for data communication between the optical information reading device 10 and other devices, and performs data communication by means of wireless communication (such as wireless LAN). In particular, the communication unit 16 of the optical information reading device 10 communicates with the host computer 19 of the information processing system 100. Note that it is sufficient for data to be communicated between the optical information reading device 10 and the host computer 19 via the communication unit 16, and it is not necessary for the communication unit 16 and the host computer 19 to communicate directly. For example, there may be a device (such as a router or gateway device) that relays communication between the optical information reading device 10 and the host computer 19.

[0029] The host computer 19 is equipped with input devices such as a keyboard and mouse, and a display device (monitor). The information processing system 100 can transmit information input by the user operating the host computer 19 to the optical information reader 10 via the communication unit 16. The information processing system 100 can also transmit information output from the optical information reader 10 to the user via the display device. The user operating the host computer 19 may be the same user as the worker carrying the optical information reader 10, or a different user.

[0030] As shown in Figure 3, the imaging module 13 includes an imaging unit 31, an AIM light irradiation unit 32, and an illumination unit 33. As previously mentioned, the imaging unit 31 includes multiple combinations of image sensors 36 and imaging optical systems 37 (camera units). The AIM light irradiation unit 32 includes a first irradiation unit 32a and a second irradiation unit 32b. The illumination unit 33 includes multiple light sources for illumination light (such as LED lighting units).

[0031] Furthermore, the imaging module 13 in Figure 3 includes a distance measuring unit 34 for measuring the distance from the optical information reading device 10 to the object 11. The distance measuring unit 34 is a unit that includes, for example, a processor for calculations and a memory for storing information. The distance between the optical information reading device 10 and the object 11 can be detected, for example, by LiDAR technology (a technology for detection and distance measurement using light). Specifically, the distance to the object 11 may be calculated, for example, based on the time of flight of the illumination light emitted by the illumination unit 33 or the AIM light 40 emitted by the AIM light illumination unit 32. Note that the distance measuring unit 34 does not necessarily have to be included in the imaging module 13 as an independent unit, and may be, for example, part of the functions realized in the control unit 50.

[0032] The control unit 50 shown in Figure 4 is a unit that controls the operation of the optical information reading device 10. The control unit 50 is connected to the output unit 12, the imaging module 13 (Figure 3), the operation unit 15, and the storage unit 60 (Figure 5). The control unit 50 includes an input / output control unit 51, a screen generation unit 52, and an information processing unit 53.

[0033] The input / output control unit 51 is an input / output interface that controls the data input to the optical information reading device 10 and the data output from the optical information reading device 10. For example, the input / output control unit 51 generates input data representing information on how the operation keys (such as the trigger key 18) included in the operation unit 15 were operated, and transmits the input data to the unit that requires it. The input / output control unit 51 also transmits data representing the symbol information read from the symbol 20 by the information processing unit 53 to the output unit 12. In addition, the input / output control unit 51 controls all input / output processing performed in the optical information reading device 10, such as transmitting images captured by the imaging unit 31 of the imaging module 13 as input images to the information processing unit 53, and transmitting commands to the imaging unit 31 to perform imaging.

[0034] The screen generation unit 52 generates screen data to be displayed on the display unit 14 based on images captured by the imaging unit 31 and the results of information processing by the information processing unit 53. For example, the screen generation unit 52 performs processing such as combining display components generated by the information processing unit 53 with images captured by the imaging unit 31.

[0035] The input / output control unit 51 and the screen generation unit 52 may be dedicated units (input / output processor, image processor, etc.) that perform input / output control processing and screen generation processing, respectively, independently of the information processing unit 53. However, the input / output control unit 51 and the screen generation unit 52 may be part of the functions implemented by the information processing unit 53.

[0036] The information processing unit 53 is a unit that includes a processor such as a CPU (Central Processing Unit). The information processing unit 53 reads and executes program data stored in the storage unit 60, thereby realizing functions such as a string processing unit 54, a relative reading processing unit 55, a matrix processing unit 56, a specified reception unit 57, a reading target setting unit 58, and a format information acquisition unit 59.

[0037] The memory unit 60 in Figure 5 is a unit such as a ROM that stores electronic data. The memory unit 60 stores a machine learning model 61, input images 62, setting images 63, operation images 64, user applications 90, and reading target setting information 76. As mentioned above, the memory unit 60 also stores program data for realizing the functions of the information processing unit 53, but for the sake of explanation, the various functions of the information processing unit 53 are shown in Figure 4 as being included in the control unit 50.

[0038] Next, the overall operation flow of the optical information reader 10 will be explained with reference to the flowchart in Figure 6. When an operator uses the optical information reader 10 to read a symbol 20, pre-configuration is performed in step S10 before the reading is executed. Pre-configuration sets various items that should be predetermined for reading the symbol 20. For example, depending on the form of the symbol 20, format information of the symbol 20 such as the string format and barcode standard type, and information regarding the arrangement of the symbol 20 such as the number of rows / columns in the table document may be registered by pre-configuration. In addition, the relative positional relationship between the aiming position of the AIM light 40 and the position of the symbol 20 to be read may also be registered by pre-configuration. The contents registered by pre-configuration are stored in the storage unit 60.

[0039] If the pre-configuration is complete, the reading mode of the optical information reader 10 is activated in step S11. For example, the reading mode may be activated by the operator operating a key for activating the reading mode included in the operation unit 15. Alternatively, the reading mode may be activated by the operator selecting an icon for activating the reading mode from the setting menu screen displayed on the display unit 14.

[0040] When the reading mode is activated, in step S12, the optical information reading device 10 irradiates the object 11 with AIM light 40. With the AIM light 40 irradiated onto the target aiming position, such as the position of the symbol 20 to be read or the position to be the center of the imaging field of view, the operator operates the trigger key 18 in step S13. Note that if the optical information reading device 10 is a smartphone type, the AIM light irradiation unit 32 may not be provided. In that case, the optical information reading device 10 may not irradiate the AIM light 40, but instead display the image generated by the imaging unit 31 as a live view on the display unit 14, and superimpose a virtual AIM that defines the aiming position on a specific position (for example, the center) on the image displayed as a live view on the display unit 14. It is preferable that the virtual AIM has the same shape as the AIM light 40. In other words, a virtual aima including a figure that mimics the first aima light 40a of linear light extending in the first direction 40H and a figure that mimics the second aima light 40b of linear light intersecting the first direction 40H should be displayed on the display unit 14. The operator can determine the aiming position by adjusting the orientation of the optical information reading device 10 (the direction the imaging unit 31 faces) to bring the virtual aima displayed on the display unit 14 closer to the symbol 20 to be read on the live view image.

[0041] When the trigger key 18 is operated, the optical information reader 10 turns off the AIM light 40 in step S14, and then in step S15, the imaging unit 31 images the object 11. Note that step S14, which turns off the AIM light 40 immediately before step S15 in which imaging takes place, is not necessarily performed. For example, the AIM light 40 may always blink (on and off repeatedly) in reading mode. In this case, step S14 is not performed, and imaging may be performed at the moment the AIM light 40 is turned off. In step S15, the imaging unit 31 images the area within the imaging field of view and generates an input image that includes the symbol 20 attached to the object 11. If a virtual AIM is displayed on the display unit 14 instead of illuminating with AIM light 40, step S14 is not performed, and the area within the imaging field of view at the time the trigger key 18 is operated is imaged.

[0042] When the object 11 is imaged, a reading process is performed on the input image obtained from the image in step S16. The details of this reading process vary depending on the form of the symbol 20 to be read. In the reading process, for example, string processing is performed to narrow down the symbol 20 that will become the output string from multiple string candidates, relative reading processing is performed to read the symbol 20 based on the relative position between the aiming position and the reading target, and matrix processing is performed to read the symbol 20 that has been attached as a matrix.

[0043] After the reading process, in step S17, it is determined whether the reading of symbol 20 was successful. If the reading of symbol 20 is successful (YES in step S17), the process proceeds to step S18, and the read symbol information is output to the storage unit 60 or the output unit 12. If the reading of symbol 20 fails (NO in step S17), the process returns to step S12, and imaging is performed again.

[0044] In step S17, whether or not the reading of symbol 20 was successful may be determined automatically or manually by the operator. As an example of automatic determination, the reading may be determined to be successful if symbol information was obtained from symbol 20 (decoding was successful or a string was obtained by OCR processing). As an example of manual determination, the display unit 14 may display the symbol information obtained by the reading process, and an option may be displayed allowing the operator to select whether or not the symbol information matches the operator's intention. If the obtained symbol information does not match the operator's intention, the operator performs an operation to indicate that the obtained symbol information is not as intended. Then the determination in step S17 becomes "NO", and the obtained symbol information is discarded. After that, the process returns to step S12, and imaging is performed again.

[0045] Next, an example of the reading process in step S16 will be explained with reference to the flowchart in Figure 7. For example, if an object 11 with multiple strings of characters attached as symbol 20 is imaged, the input image obtained by the image will contain multiple characters. In step S21, which is included in the reading process, the optical information reading device 10 performs character detection processing to detect the multiple characters contained in the input image.

[0046] Next, in step S22, the optical information reading device 10 performs character concatenation to obtain multiple string candidates from the input image based on these multiple characters. Subsequently, in step S23, the optical information reading device 10 performs string filtering to narrow down the string candidates to the output string to be output. Here, the output string to be output is a string that should be output to a device and component other than the control unit 50 that performs the reading process. For example, a string that should be displayed on the display unit 14 or a string that should be sent to the host computer 19, etc., is an output string that should be used for processing through the output unit 12. Also, a string that should be output to and stored in the storage unit 60 may be an output string. If, as a result of filtering, there is only one output string, the information represented by that output string becomes the symbol information obtained by the reading process. If there are multiple output strings, the information represented by any one of them becomes the symbol information obtained by the reading process. Which output string's information is read as symbol information may be manually selected by the operator, or the optical information reading device 10 may select it automatically.

[0047] Figure 8 shows an example of the procedure for narrowing down the output string 29. When the imaging unit 31 generates an input image containing multiple strings, the input image contains multiple characters as symbols 20, as shown in Figure 8. For the input image containing multiple characters, the string processing unit 54 (Figure 4) of the control unit 50 performs string reading processing (string processing). First, the string processing unit 54 uses the machine learning model 61 (Figure 5) stored in the memory unit 60 to detect the multiple characters contained in the input image (performs character detection).

[0048] The machine learning model 61 is an input / output model that takes an input image as input and detects multiple characters contained in that image. For example, the machine learning model 61 could be an input / output model that has been trained to take an image containing various characters as input and output information about the characters contained in that image. The character information output by the machine learning model 61 should ideally include information about the position of each character in the input image (such as coordinate data), the size of each character, and information about the type of each character (such as character code).

[0049] After character detection is performed by the machine learning model 61, the string processing unit 54 (Figure 4) obtains multiple string candidates 21 from the input image based on the multiple characters detected by the machine learning model 61 (character concatenation). In concatenation, the string processing unit 54 groups together multiple characters that are estimated to be highly related to each other as string candidates 21, based on the relative positions of the characters, the relationship between the characters' sizes, etc. For example, if multiple characters of similar size are arranged horizontally at small intervals in the input image, and are positioned vertically at similar intervals, those characters are estimated to be highly related to each other. Specifically, as shown in Figure 8, the string processing unit 54 concatenates a series of horizontally arranged characters into a single string candidate 21.

[0050] The string processing unit 54 performs a process to narrow down the output string from multiple string candidates 21 obtained by string concatenation. If only one string candidate 21 is obtained, that string candidate 21 becomes the output string. Multiple indicators may be used as criteria for narrowing down the output string. Examples of indicators include string likelihood, distance to the targeting position in the input image, and degree of match with the format pattern. It is sufficient that at least one of the above indicators is used, and it is not necessary to use multiple indicators. For example, the output string may be narrowed down based only on the distance to the targeting position.

[0051] For example, Figure 8 shows how, among multiple string candidates 21, the second string candidate 21, which is closest to the targeting position, is narrowed down as the output string candidate. The targeting position is the position determined by the AIM light 40, but since the AIM light irradiator 32 is turned off at the moment the input image is captured, the AIM light 40 is not included in the input image. The optical information reading device 10 calculates the position where the AIM light 40 was irradiated just before the image was captured and treats that position as the targeting position corresponding to the AIM light 40 in the input image. Although the AIM light 40 is not included in the input image, for explanatory purposes, the AIM light 40 will be shown in the diagram below. Also, the targeting position may be a position determined by a virtual AIM on the display unit 14, rather than the AIM light 40 from the AIM light irradiator 32. Both the virtual AIM and the AIM light 40 may be collectively referred to as AIM light 40 below.

[0052] The optical information reading device 10 can calculate the position where the AIM light 40 is irradiated onto the object 11 (or the position where the virtual AIM is displayed) as the aiming position, based on the distance between the optical information reading device 10 and the object 11, and the tilt of the optical information reading device 10. The distance between the optical information reading device 10 and the object 11 is measured by the distance measuring unit 34 of the imaging module 13. The tilt of the optical information reading device 10 can be detected, for example, by an acceleration sensor located inside the housing 17.

[0053] Alternatively, the imaging unit 31 may capture both an input image that does not contain AIM light 40 and a reference image that contains AIM light 40, and the targeting position may be calculated by comparing the input image and the reference image.

[0054] The string likelihood, which is one of the indicators used by the string processing unit 54 to narrow down the output strings, is an indicator that shows the likelihood (plausibility) of a string. The string processing unit 54 calculates the string likelihood, for example, by using the results of evaluations of each of the multiple string candidates 21 based on predetermined likelihood conditions.

[0055] The string processing unit 54 calculates the string likelihood based on at least one of the following as likelihood conditions: for example, a score indicating character resemblance output by the machine learning model 61, the variation in font style for each character included in the string candidate 21, the string margin, reading history information, and date-related information.

[0056] The character resemblance score output by the machine learning model 61 is part of the information returned by the machine learning model 61 as output when it receives an input image. It is a numerical value that indicates the degree of certainty with which each of the multiple characters detected by the machine learning model 61 is detected as a character. The higher this score, the more likely it is that the detected characters are correct natural language characters. Therefore, string candidates 21 composed of characters with high character resemblance scores are highly likely to be correct natural language strings, and thus a high string likelihood is calculated.

[0057] The variation in font style for each character in the candidate string 21 refers to the variation in at least one of the following: spacing, height, or contrast for each character. It is highly likely that each of multiple characters belonging to the same string will have consistent spacing, height, and contrast. Therefore, candidate strings 21 with low variation in spacing, height, and contrast for each character are more likely to be the string desired by the operator as the output string, resulting in a high string likelihood. For example, in Figure 8, the part of candidate string 21 in the second line that includes the numbers in the upper right, "2024.07.04", has a different character height compared to the rest of the string. Therefore, this part is less likely to be the output string, and is excluded from candidate string 21.

[0058] The string margin is the size of the blank space before and after the sequence of characters contained in the string candidate 21. For the string candidate 21 "2025.11.19" shown in Figure 9, there is another string 27, "Expiration Date," to its left and a border line 28 to its right. In this case, the string processing unit 54 searches from the end of the string candidate 21 along the direction of the character sequence, and the distance to the position where the other string 27 or border line 28 is found is the string margin. For example, when the string processing unit 54 performs a search along the direction of the character sequence, it is preferable to recognize the position where a certain level of contrast change occurs in the input image as the position where the other string 27 or border line 28 exists. In this case, in addition to the position where the ruled lines are actually drawn on the object 11, the edges of the object 11 to which the symbol 20 is attached, such as the edge of the paper, are also recognized as border lines 28. In Figure 9, the blank space between candidate string 21 and the other string 27 to its left is the left string margin ΔL, and the blank space between candidate string 21 and the right border 28 is the right string margin ΔR. If these string margins are sufficiently large (larger than a predetermined margin threshold), candidate string 21 can be considered a single string independent of the other strings 27, and therefore the string likelihood is calculated to be high.

[0059] Reading history information refers to information about strings that have been read in the past. The storage unit 60 of the optical information reading device 10 stores reading history information about strings that have been read in the past. The string processing unit 54 compares the string candidate 21 with the reading history information and calculates a higher string likelihood the more similar the string candidate 21 is to strings that have been read in the past (past strings). It is desirable to consider the similarity of past strings, especially those read in the most recent period (for example, within 24 hours). For example, in a situation where you want to sequentially read multiple workpieces (objects) that have 10-digit strings attached to them, if each workpiece contains multiple 10-digit strings, it becomes easier to identify which string should be narrowed down as the output string based on the reading history information.

[0060] In addition to the character-likeness score, font variability, string margins, and reading history information, other factors may be considered when calculating string likelihood. For example, if the string to be read is a date string, it is good to consider information related to the date. Specifically, it is good to consider whether the string contains separators to separate the year, month, and day, and whether the string contains kanji or English words that represent the year, month, and day. It is also good to consider comparison information between the date (time) indicated by the date string to be read and the current time, and if there are multiple date strings, the time series between the dates represented by each date string is also good to consider. For example, the manufacturing date is in the past compared to the current time, and the expiration date and best-before date are in the future compared to the current time. Also, in the time series, the expiration date is later than the manufacturing date. By considering this information related to the date, depending on the type of date to be read, it becomes easier to reliably narrow down the output string to the desired date.

[0061] The degree of match with the format pattern, which is one of the indicators used by the string processing unit 54 to narrow down the output string, is the degree of match between the string candidate 21 and a format pattern predetermined as the format of a string that can become the output string. The format pattern should be set in advance by the operator before the string is read using the optical information reading device 10, indicating what format the string to be read should be.

[0062] For example, in Figure 8, if the format pattern is set to "9-digit number" (a string of digits arranged nine times in a row), the "Lot No:" part of the string candidates 21 that does not match the format pattern is excluded from the string. Then, "123456789", which matches the format pattern, is narrowed down to the output string 29.

[0063] Here, the degree of match with the format pattern may simply be a numerical value indicating "whether or not it matches," or a numerical value indicating the degree of match with the format pattern may be used. For example, the numerical values ​​indicating "whether or not it matches" could be "1" for "match" and "0" for "does not match." For example, the numerical values ​​indicating the degree of match with the format pattern could be a number between "0" and "1."

[0064] When the degree of matching with a format pattern is used as one of the indicators for narrowing down the output string 29, it is desirable to determine, for example, whether the degree of matching exceeds a predetermined matching threshold. For example, if the matching threshold is set to "0.5", even if it does not perfectly match the format pattern, if it matches to some extent, it may become the output string 29. On the other hand, if the matching threshold is set to "0.9", almost all string candidates other than the one that perfectly matches the format pattern 21 will be excluded.

[0065] When the string processing unit 54 uses multiple indicators such as string likelihood, distance to the target position, and degree of match with the format pattern as criteria for narrowing down the output string 29, it is desirable that the output string 29 be narrowed down by comprehensively evaluating these indicators. For example, it is desirable that a composite score be calculated based on multiple indicators, and that the string candidate 21 with the highest composite score be narrowed down as the output string 29.

[0066] On the other hand, these indicators may be given priority. For example, the string processing unit 54 may first calculate the degree of match with the format pattern for each string candidate 21 and determine whether the degree of match exceeds a match threshold. Then, the string processing unit 54 may narrow down the output string 29 from the string candidates 21 whose degree of match with the format pattern exceeds the match threshold (matches the format pattern or has a high degree of match) based on at least one of the string likelihood and the distance to the target position.

[0067] Furthermore, a priority order may be assigned between the string likelihood and the distance to the targeting position. For example, if there are multiple string candidates 21 whose degree of matching with the format pattern exceeds the matching threshold, the string likelihood should be calculated for those string candidates 21 next. A likelihood threshold should be predetermined for the string likelihood. The likelihood threshold is a number that indicates that if the string likelihood exceeds this number, there is a high probability that the string candidate 21 is appropriate as the string to be read. If there are multiple string candidates 21 whose degree of matching with the format pattern exceeds the matching threshold and whose string likelihood also exceeds the likelihood threshold, the string processing unit 54 should select the string candidate 21 with the shortest distance to the targeting position as the output string 29.

[0068] Furthermore, instead of sequentially narrowing down the candidates using string likelihood and then narrowing down the candidates using distance to the target position, the string processing unit 54 may narrow down the candidates using both string likelihood and distance to the target position in a comprehensive manner. For example, the string processing unit 54 may calculate a composite score based on string likelihood and distance to the target position for string candidates 21 whose degree of matching with the format pattern exceeds the matching threshold. Hereinafter, the composite score calculated based on string likelihood and distance to the target position will be called the string position likelihood. The string position likelihood is calculated, for example, by a function that takes string likelihood and distance to the target position as variables. The string position likelihood is higher the higher the string likelihood of the string candidate 21 and the shorter the distance to the target position. The string processing unit 54 may select the string candidate 21 with the highest string position likelihood among the string candidates 21 whose degree of matching with the format pattern exceeds the matching threshold as the output string 29.

[0069] Note that the priority order of the metrics used to narrow down the output string 29 is not limited to the above order: degree of match with the format pattern, string likelihood, and distance to the targeting position. For example, distance to the targeting position may have the highest priority.

[0070] For example, the string processing unit 54 may sort the multiple string candidates 21 in descending order of their distance to the target position. Then, for each of the sorted string candidates 21, the string processing unit 54 determines, in sort order, whether the degree of matching with the format pattern exceeds a predetermined matching threshold, and whether the string likelihood exceeds a predetermined likelihood threshold. If, as a result of the determination, a string candidate 21 is found whose degree of matching with the format pattern exceeds the matching threshold and whose string likelihood exceeds the likelihood threshold, the string processing unit 54 may select that string candidate 21 as the output string 29. For example, in Figure 8, it can be said that the output string 29 is narrowed down using this procedure.

[0071] Next, referring to Figure 10, the setting of the format pattern and margin threshold will be explained. The optical information reading device 10 is preferably equipped with a function for setting the format pattern and margin threshold as part of the pre-configuration. For example, when the setting screen key included in the operation unit 15 (Figure 1) is operated, a setting menu screen (not shown) for making various settings of the optical information reading device 10 is displayed on the display unit 14. This setting menu screen includes an item called "Format Registration," and when the "Format Registration" item is selected by the operator, the format registration screen 38 shown in Figure 10 is displayed on the display unit 14 (or the monitor of the host computer 19).

[0072] On the format registration screen 38, the format pattern and margin threshold are set by the operator. The format registration screen 38 includes a format registration field 38a, a left margin threshold field 38b, a right margin threshold field 38c, and a detailed settings button 38d.

[0073] The format registration field 38a registers a format pattern indicating the format of a string that can become the output string 29 to be output. In FIG. 10, the pattern "AA-?99" is registered as the format pattern. In this format registration field 38a, "A" represents an alphabet, "-" (hyphen) represents a delimiter character, and "9" represents a number. And "?" represents any character (All) used in the string. Therefore, the pattern "AA-?99" represents a string in which characters are arranged in the order of two alphabets, a delimiter character, any character, and two numbers. Note that the symbol representing any character is not limited to "?". The symbol representing any character is preferably a character different from the characters assumed to be the reading target, but it may be sufficient as long as the user can understand that the symbol represents any character. For example, the Chinese character "全" may be used as a symbol representing any character, or a unique symbol in which the string "All" is contained within a one-character frame may be used.

[0074] The operator can change the character type displayed in the format registration field 38a by operating the up arrow key and down arrow key (not shown) included in the operation unit 15. That is, the operator can set any arrangement of alphabet, number, any character, and delimiter character as the format pattern.

[0075] Also, the operator can determine the margin threshold related to the blank area (string margin) before and after the character arrangement of the string candidate 21 by the left margin threshold field 38b and the right margin threshold field 38c. For example, the operator can set the thresholds of the left string margin ΔL and the right string margin ΔR by operating such as inputting a numerical value into the left margin threshold field 38b and the right margin threshold field 38c or selecting a numerical value from a pull-down list. Also, the operator can set the margin threshold to "none". By setting the margin threshold to "none", there is a possibility that the string candidate 21 without margins on the left and right may be narrowed down as the output string 29.

[0076] If the settings on the format registration screen 38 are made as shown in Figure 10, then, for example, the string "ZX-235" will be determined to be a string that matches the format pattern. Furthermore, if the distance from the beginning and end of the string to the border lines 28, i.e., the left string margin ΔL and the right string margin ΔR, exceed the set margin thresholds, then "ZX-235" will be filtered as the output string 29.

[0077] When the detailed settings button 38d on the format registration screen 38 is operated, the display on the display unit 14 transitions to a detailed settings screen (not shown) where the detailed settings of the format registration screen 38 can be changed. On the detailed settings screen, for example, it is preferable to select the type of characters to be used in the string. For example, if the string to be read uses kanji characters in addition to alphabets, numbers, and delimiters, it is preferable to set it so that kanji characters can also be registered in the format pattern.

[0078] Furthermore, it would be beneficial to allow users to register characters that are considered delimiters in the detailed settings screen. For example, it would be good to be able to register not only "-" (hyphen) but also other characters that can be used as delimiters, such as " / " (slash), "," (comma), "." (dot), and ":" (colon). It would also be good to be able to register spaces at regular intervals as delimiters. It would also be good to be able to set whether "arbitrary characters" should be characters other than delimiters, or whether delimiters should be included in "arbitrary characters". In addition, it would be good to be able to set a threshold for the contrast change that determines the input image to be frame line 28 in the detailed settings screen.

[0079] Next, with reference to Figure 11, a specific example of how output strings 29 are narrowed down based on the distance to the target position of the string candidate 21 will be described. Figure 11 shows an example of an object 11 with multiple strings attached. The object 11 in Figure 11 is a cylindrical product (for example, a food container), and multiple strings are attached to its bottom as symbols 20. A transparent film and seal are attached to the bottom of the object 11 to cover the symbols 20, but the optical information reading device 10 can read the symbols 20 through the film and seal.

[0080] The string of characters as symbol 20 in Figure 11 includes a date string representing the "expiration date" and a string of numbers indicating information about the "manufacturing date". If the operator intends the date string for the "expiration date" to be the output string 29, the optical information reader 10 is directed so that the AIM light 40 is shone near the date string for the "expiration date".

[0081] The input image obtained by imaging the bottom of the object 11 includes a date string representing the "expiration date" and a numerical string indicating information about the "manufacturing date". In this case, the date string representing the "expiration date" is at a shorter distance to the targeting position (the position illuminated by the AIM light 40). The string processing unit 54 (Figure 4) of the optical information reading device 10 acquires these strings from the input image as string candidates 21. Based on the distance to the targeting position, the string processing unit 54 narrows down the strings from the string candidates 21 to the date string representing the "expiration date" which is at a shorter distance to the targeting position, and selects it as the output string 29.

[0082] Next, referring to Figure 12, we will explain the case in which the string candidate 21 with the highest string likelihood becomes the output string 29. In Figure 12, the object 11 is a product shipping slip. This shipping slip has multiple strings attached to it. In Figure 12, as in Figure 10, a predetermined format pattern is assumed to be a string pattern ("AA-?99") in which characters are arranged in the order of two alphabet characters, a delimiter, an arbitrary character, and two digits.

[0083] In Figure 12, the EIM light 40 is shone near the string candidate 21 "BQ-S65783" which represents the "product number". Of these string candidates 21, the string fragment 24 of "BQ-S65" matches the format pattern. On the other hand, the string "ZX-235" which represents the "shipping destination" and is located far from the aiming position of the EIM light 40, not only matches the format pattern but also has sufficiently large string margins in the direction of character arrangement. Therefore, the string likelihood of "ZX-235" is higher than the string likelihood of the string fragment 24 of "BQ-S65". Here, when the output string 29 is narrowed down based on the combined score calculated based on the string likelihood and the distance to the aiming position, if the combined score of the string "ZX-235" is higher than that of the string fragment 24 of "BQ-S65", then the string "ZX-235" is narrowed down as the output string 29.

[0084] Next, referring to Figure 13, we will explain the case in which the output string 29 is narrowed down based on the degree of match with the format pattern. In Figure 13, it is assumed that a format pattern of 4 digits, a delimiter, 2 digits, a delimiter, and 2 digits ("9999.99.99") is predetermined for reading a date string representing a date. In Figure 13, the EIM light 40 is shone near the string candidate 21 "N42108". Among the string candidates 21 close to this aiming position, the string fragment 24 "2108" is a string that is valid as a date (August 2021), but the degree of match with the format pattern is low. On the other hand, the string "2026.08.27", which is located far from the aiming position of the EIM light 40, has a high degree of match with the format pattern. In this case, regarding the degree of matching with the format pattern, if the string fragment 24 "2108" does not exceed the matching threshold, but the string "2026.08.27" does, then the string "2026.08.27" is narrowed down as the output string 29. Furthermore, if the calculation criteria for string likelihood consider whether or not the date string contains a delimiter, then "2026.08.27", which contains a separator, is given priority as the output string 29 from the perspective of string likelihood as well.

[0085] Next, referring to Figure 14, we will explain the case where the output string 29 is narrowed down based on the distance to the aiming position. In Figure 14, it is assumed that the same format pattern as in Figure 13 is predetermined for reading date strings. In Figure 14, there are two strings that can be date strings. For example, the upper string "2025.05.03" represents the product's manufacturing date, and the lower string "2026.05.02" represents the product's expiration date. If the string that the operator wants to read is the lower string representing the expiration date, the AIM light 40 is shone near the lower string. In this case, the upper string (string candidate 21) and the lower string have a similar degree of matching with the format pattern, and both exceed the matching threshold. Also, the upper string and the lower string have similar string likelihoods, and both exceed the predetermined likelihood threshold. In this case, the string with the shortest distance to the aiming position is the lower string, so the string processing unit 54 can narrow down the output string to the lower string "2026.05.02". Furthermore, the string processing unit 54 can also narrow down the output string 29 using the string position likelihood of a composite score calculated based on the string likelihood and the distance to the targeting position. The upper and lower strings have similar degrees of match with the format pattern and string likelihood, but the string position likelihood is higher for the lower string, which is closer to the targeting position. Therefore, the string processing unit 54 can narrow down the output string 29 to the lower string "2026.05.02", which has the highest string position likelihood.

[0086] Next, as another example of string processing, we will describe processing using the AI ​​processing core 53a and multiple image processing cores 53b corresponding to the AI ​​processing core 53a in the information processing unit 53 (Figure 4). Figure 15 schematically shows the correspondence between the AI ​​processing core 53a and the image processing core 53b included in the information processing unit 53.

[0087] The information processing unit 53 (processor such as a CPU) of the control unit 50 preferably has an AI processing core 53a and multiple image processing cores 53b corresponding to the AI ​​processing core 53a, as shown in Figure 15. Although multiple image processing cores 53b correspond to one AI processing core 53a, the number of AI processing cores 53a is not limited to one, and there may be multiple combinations of AI processing cores 53a and image processing cores 53b. In other words, it is preferable that there be multiple AI processing cores 53a, and that multiple image processing cores 53b correspond to each of the AI ​​processing cores 53a. The string processing unit 54 included in the information processing unit 53 also has these AI processing cores 53a and image processing cores 53b.

[0088] The AI ​​processing core 53a of the string processing unit 54 performs character detection processing using the machine learning model 61 (Figure 5). Specifically, the AI ​​processing core 53a inputs an input image to the machine learning model 61 (artificial intelligence model, AI model). The machine learning model 61 detects multiple characters contained in the input image and outputs information about the detected multiple characters. The AI ​​processing core 53a sends the output of the machine learning model 61, that is, the information about the detected multiple characters, to multiple image processing cores 53b. This character detection processing by the AI ​​processing core 53a is executed at high speed.

[0089] The image processing core 53b performs character concatenation and calculates string likelihood. First, based on information about multiple characters detected by the AI ​​processing core 53a, the image processing core 53b obtains multiple string candidates 21 by grouping multiple characters that are estimated to be highly related to each other based on their relative positions and size relationships. Note that the character concatenation process that obtains multiple string candidates 21 based on these detected characters may also be performed by the AI ​​processing core 53a.

[0090] The image processing core 53b calculates the string likelihood for each of the acquired string candidates 21 based on predetermined likelihood conditions. These likelihood conditions may include, for example, a score indicating character resemblance, variations in font style for each character, string margins, and reading history information.

[0091] Processing by the image processing core 53b may take longer than processing by the AI ​​processing core 53a. Therefore, as shown in Figure 16, the overall processing speed of the optical information reading device 10 is improved by performing parallel processing with multiple AI processing cores 53a. Figure 16 shows the AI ​​processing core 53a and the image processing core 53b operating in parallel.

[0092] As an example, as shown in Figure 16, the imaging unit 31 may contain multiple camera units, each having an image sensor 36 and an imaging optical system 37 (in this case, three cameras: Camera A, Camera B, and Camera C). The input images captured by each camera unit are sequentially input to the AI ​​processing core 53a. If there are multiple AI processing cores 53a, the input images from each camera unit may be input to each AI processing core 53a in parallel.

[0093] Once character detection from the input image of the first camera unit (camera A) is complete, information about the detected characters is input to the image processing core 53b. The character information is sent to the image processing core 53b that is not currently performing any other processing (in this case, image processing core A) among the multiple image processing cores 53b (there are three image processing cores A, B, and C in Figure 16).

[0094] While image processing core A performs character concatenation and calculates string likelihood, AI processing core 53a performs character detection from the input image of the second camera unit (camera B). Therefore, the processing of image processing core A with respect to the input image of camera A and the processing of AI processing core 53a with respect to the input image of camera B are executed in parallel.

[0095] While character detection by the AI ​​processing core 53a is processed quickly, the processing by the image processing core A takes longer than that of the AI ​​processing core 53a. Therefore, by the time character detection from the input image of camera B is complete, the processing by image processing core A may not be finished. In this case, the AI ​​processing core 53a sends information about the multiple characters detected from the input image of camera B to the other image processing core 53b (image processing core B in Figure 16), which is not currently processing.

[0096] As a result, the processing of the input image from camera A by image processing core A and the processing of the input image from camera B by image processing core B are executed in parallel. Similarly, the AI ​​processing core 53a sends information about multiple characters detected from the input image from camera C to image processing core C, thereby enabling the processing of image processing cores B and C to be executed in parallel.

[0097] As described above, the AI ​​processing core 53a transmits information about the multiple detected characters to multiple image processing cores 53b, allowing the character concatenation and string likelihood calculation processes for multiple input images captured by multiple camera units to be executed in parallel. This enables the AI ​​processing core 53a and the other image processing cores 53b to operate in parallel while one of the image processing cores 53b is performing the character concatenation and string likelihood calculation processes, thereby improving the overall processing speed of the optical information reading device 10.

[0098] The optical conditions (distance, angle, illumination brightness, etc.) of the object 11 relative to the optical information reading device 10 vary. Therefore, in order to correctly read the symbol information of the symbol 20 (in this case, the output string 29), it may be necessary to repeatedly acquire and read multiple input images with different imaging conditions. For example, input images are captured by multiple camera units with different focal lengths, and the reading process is performed for each input image. Here, parallel processing is performed by the AI ​​processing core 53a and the image processing core 53b, allowing the reading process for multiple input images to be executed at high speed.

[0099] Furthermore, even with only one camera unit, the reading process is accelerated by parallel processing by the AI ​​processing core 53a and the image processing core 53b. For example, if one camera unit captures images multiple times, and the captured input images are sequentially input to the AI ​​processing core 53a, then even in this case, parallel processing by the AI ​​processing core 53a and the image processing core 53b allows for parallel execution of image capture by the camera unit, character detection by the AI ​​processing core 53a, and character concatenation and string likelihood calculation by the image processing core 53b, thereby improving the overall reading speed.

[0100] Next, as another example of string processing, we will describe the case where a two-stage machine learning model 61 (Figure 5) is used. As shown in Figure 5, the machine learning model 61 may include a first model 61a and a second model 61b which has more parameters than the first model 61a. The machine learning model 61 may also include many more models with different numbers of parameters.

[0101] In string processing, models with a large number of parameters (heavyweight models) generally offer higher detection accuracy than models with fewer parameters (lightweight models), but tend to take longer to process. Therefore, the optical information reading device 10 achieves both high-precision reading and high-speed processing by using the first model 61a and the second model 61b in combination. The flowcharts shown in Figures 17 and 18 illustrate the flow of string processing using the first model 61a and the second model 61b. Figure 18 shows a part of the string processing shown in Figure 17.

[0102] In step S30 of Figure 17, the string processing unit 54 first performs character detection using the first model 61a to detect multiple characters contained in the input image. Subsequently, in step S31, the string processing unit 54 performs character concatenation to obtain multiple string candidates 21 from the input image based on the multiple characters detected by the first model 61a.

[0103] Then, in step S32, the string processing unit 54 calculates the string likelihood for each of the acquired string candidates 21 based on the character-likeness score, the variation in font for each character, the string margin, and the reading history information. Furthermore, in step S33, the string processing unit 54 calculates a combined score, the string position likelihood, for each of the string candidates 21 based on the string likelihood and the distance to the target position.

[0104] Once the string position likelihood is calculated, the string processing unit 54 determines in step S34 whether there is a valid string (valid string) among the multiple string candidates 21 that can become the output string 29. The determination of whether a valid string exists can be made, for example, by determining whether there is a string candidate 21 with a sufficiently high string position likelihood (whether there is a string candidate 21 that exceeds a predetermined threshold).

[0105] If no valid string exists (NO in step S34), the string processing unit 54 proceeds to step S35 and stores information (such as a flag) in the storage unit 60 indicating that the acquisition result of the string candidate 21 using the first model 61a was "failure". After that, the string processing unit 54 proceeds to step SA and acquires the string candidate 21 using the second model 61b. Details of the processing using the second model 61b will be described later with reference to Figure 18.

[0106] On the other hand, if a valid string exists (YES in step S34), the string processing unit 54 proceeds to step S36 and stores information in the storage unit 60 indicating that the acquisition result of the string candidate 21 using the first model 61a was "successful". After that, the string processing unit 54 proceeds to step S37 and narrows down the output string 29. If there are multiple string candidates 21, the string processing unit 54 narrows down the output string 29 from the multiple string candidates 21 based on the string likelihood, distance to the target position, degree of match with the format pattern, etc.

[0107] Next, in step S38, the string processing unit 54 calculates a string confidence score, which is an index for determining whether the narrowed-down output string 29 has a reliability worthy of being output. The string confidence score is an index that indicates the appropriateness of the output string 29, and is calculated by considering the reliability of the input image itself, in addition to the same calculation criteria as the likelihood condition of the string likelihood (such as a score indicating character resemblance). For example, if the lighting conditions during imaging by the imaging unit 31 are inappropriate and the character contrast (difference in brightness between the character part and the background part) in the input image is insufficient, the string confidence score will be low. In addition, if the input image itself is unreliable, such as when halation occurs in the input image due to reflection of lighting, the string confidence score may be low even if the string likelihood score is high.

[0108] Next, in step S39, the string processing unit 54 determines whether the calculated string confidence level is sufficiently high (for example, whether it exceeds a predetermined confidence threshold). If the string confidence level is low (NO in step S39), the string processing unit 54 proceeds to step SA and obtains the string candidate 21 using the second model 61b described later.

[0109] On the other hand, if the string confidence level is sufficiently high (YES in step S39), the string processing unit 54 proceeds to step S40 and, assuming that the string reading from the input image was successful, notifies the external system of the reading result in step S41. For example, the information of the read output string 29 is displayed on the display unit 14 of the output unit 12, transmitted to the host computer 19 via the communication unit 16, or stored in the storage unit 60.

[0110] On the other hand, if the acquisition of string candidates 21 using the first model 61a fails, or if the string confidence level of the narrowed-down output string 29 is low, the acquisition of string candidates 21 using the second model 61b is performed. Figure 18 shows a flowchart illustrating the process of acquiring string candidates 21 using the second model 61b, which is part of the string processing.

[0111] From step S42 to step S45 in Figure 18, the string processing unit 54 performs character detection, character concatenation, string likelihood calculation, and string position likelihood calculation using the second model 61b, in the same manner as when using the first model 61a.

[0112] Then, in step S46, it is determined whether or not a valid string exists among the multiple string candidates 21 obtained using the second model 61b. The second model 61b has more parameters than the first model 61a and can detect characters that are difficult to detect with the first model 61a. Therefore, even if a valid string cannot be obtained when obtaining string candidates 21 using the first model 61a, a valid string may be obtained using the second model 61b.

[0113] If a valid string exists (YES in step S46), the string processing unit 54 proceeds to step S47 and stores information in the storage unit 60 indicating that the acquisition result of the string candidate 21 using the second model 61b was "successful". Subsequently, from steps S48 to S49, the string processing unit 54 performs the processing of narrowing down the output string 29 and calculating the string confidence level.

[0114] Next, in step S50, the string processing unit 54 determines whether the calculated string confidence level is sufficiently high. If the string confidence level is low (NO in step S50), the string processing unit 54 proceeds to step S56, described below, and compares the string confidence level calculated by the first model 61a with the string confidence level calculated by the second model 61b.

[0115] On the other hand, if the string confidence level is sufficiently high (YES in step S50), the string processing unit 54 proceeds to step SY, merges with steps S40 and S41 in Figure 17, and notifies the external system of the successful reading result, indicating that the string reading from the input image was successful.

[0116] On the other hand, if no valid string exists in step S46 (NO in step S46), the string processing unit 54 proceeds to step S51 and stores information in the storage unit 60 indicating that the acquisition result of the string candidate 21 using the second model 61b was "failure".

[0117] Subsequently, the string processing unit 54 proceeds to step S52 and determines whether or not information indicating that the acquisition result of the string candidate 21 using the first model 61a was "successful" is stored in the storage unit 60.

[0118] If the determination in step S52 is "NO", that is, if no valid string candidate 21 is obtained using either the first model 61a or the second model 61b, the string processing unit 54 proceeds to step S53 and determines that the string reading has failed.

[0119] If reading the string fails, the string processing unit 54 proceeds to step SN and merges with step S41 in Figure 17 to notify that "reading failed" as the reading result. For example, information indicating "reading failed" may be displayed on the display unit 14, transmitted to the host computer 19, or stored in the storage unit 60.

[0120] On the other hand, if the determination in step S52 is "YES", or if the determination in step S50 is "NO", the string processing unit 54 proceeds to step S56 and compares the string confidence score from the first model 61a with the string confidence score from the second model 61b.

[0121] In step S56, either the first model 61a or the second model 61b has succeeded in obtaining the string candidate 21, or both models have succeeded in obtaining the string candidate 21, but the string confidence level of the output string 29 is low.

[0122] Based on a comparison of the string confidence scores obtained by the first model 61a and the second model 61b, the string processing unit 54 proceeds to step SY as the read result, selecting the output string 29 from the model with the higher string confidence score, and notifies the external system of the reading result. If only one of the first model 61a or the second model 61b successfully obtains the string candidate 21, the output string 29 from the model that successfully obtained the candidate should be considered to have a higher string confidence score. Furthermore, the reading result notified after the string confidence score comparison may include information indicating that the string confidence score is low.

[0123] As described above, when string processing is performed using both the first model 61a and the second model 61b, the first model 61a can process at high speed. Therefore, if the first model 61a can obtain a reading result with high string reliability, the processing will be completed quickly.

[0124] On the other hand, the second model 61b, which has more parameters, takes longer to process than the first model 61a, but it can read with high accuracy. For example, as shown in Figure 11, characters that are directly attached to the surface of an object and then covered with a film are difficult to read, and the lightweight first model 61a may not be able to read them well. By using the second model 61b, which is a weight model, it becomes possible to read them correctly. In this way, things that can be read by the first model 61a are read at high speed, and strings that are difficult to read are read by the second model 61b, thus achieving both high speed and high accuracy in reading.

[0125] Furthermore, if machine learning model 61 includes many more models (three or more) with different numbers of parameters, these models can be used in combination to maintain high speed for strings with low readability while also being able to handle strings with higher readability.

[0126] Furthermore, when narrowing down the output string 29, it is also advisable to refer to the direction in which the candidate string 21 extends (the direction in which the characters are arranged). Figure 19 is a diagram illustrating string narrowing based on the direction in which the candidate string 21 extends. As shown in Figure 19, we will explain the case where the object 11 (not shown in Figure 19) has the string "ABCD" which is the candidate string 21, and the string "EFGH" which the operator wants to be the output string 29. Here, the string "ABCD" and the string "EFGH" have different character directions.

[0127] When multiple strings with different character orientations exist, the operator irradiates the strings that they wish to be output string 29 with the first direction 40H of the first AIM light 40a of the AIM light 40 so as to match as possible.

[0128] The string processing unit 54 calculates the direction of the characters, that is, the direction in which the string candidate 21 extends, based on the position data and character size of each character included in the string candidate 21. The string candidate 21 then compares the direction in which the string candidate 21 extends with the direction of the first AEMA light 40a included in the AEMA light 40.

[0129] The string processing unit 54 calculates the degree of agreement between the direction in which the string candidate 21 extends and the first direction 40H (specifically, the smallness of the angle θ between the first direction 40H and the direction of character arrangement). The string processing unit 54 then narrows down the output string 29 based on the distance to the target position of the string candidate 21 and the degree of directional agreement. For example, a combined score of the distance to the target position and the degree of directional agreement is calculated.

[0130] In Figure 19, the string "ABCD" is closer to the targeting position (the position corresponding to the AIM light 40) than the string "EFGH", but the string "EFGH" has a higher degree of matching in the direction of character arrangement with respect to the first direction 40H (smaller angle). A string with a high degree of matching with the first direction 40H is more likely to be the output string 29 that the operator wants to read. In this case, the string "EFGH" will have a higher composite score, and the string "EFGH" will be narrowed down as the output string 29.

[0131] In this way, when narrowing down the output string 29, the direction in which the string candidates 21 extend is referenced, so that if there are multiple strings with different character orientations, the string with the orientation desired by the operator is narrowed down as the output string 29.

[0132] Next, we will describe another example of the reading process in step S16 of Figure 6. Hereafter, we will describe a case where the object 11 includes multiple types, such as the first type object 11a shown in Figure 20 and the second type object 11b shown in Figure 21. However, in the following, the first type object 11a and the second type object 11b may be collectively referred to as object 11 as needed. Figure 20 shows a first type object 11a with multiple symbols 20 attached. A first type object 11a is, for example, an inbound receipt in warehouse management.

[0133] The Type 1 object 11a in Figure 20 is marked with multiple strings of characters and a barcode 23 as symbols 20. The Type 1 object 11a is marked with multiple strings of characters that indicate symbolic information about the product received into the warehouse, such as the name of the "variety," the name of the "product," the date on which "processing" was performed, the date of "expiration date," the name of the "country of origin," the "identification number," and the "lot number." The barcode 23 is an encoded representation of symbolic information different from the strings (for example, the price of the product). The symbolic information represented by the barcode 23 may include the same information as the information represented by the strings. In addition to the barcode 23, the Type 1 object 11a may also include multiple symbols 20 other than strings, such as two-dimensional codes.

[0134] When a first-class object 11a, which has multiple symbols 20 attached in this manner, is to be read by the optical information reading device 10, the operator may wish to read two or more symbols 20. For example, in managing the expiration dates of products stored in a warehouse, information on the product's "identification number" and its corresponding "expiration date" may be required.

[0135] In this case, the worker, for example, wishes to read the numbers "123456" representing the "identification number" and irradiates the AIM light 40 to a position close to the "identification number" (or adjusts the position of the virtual AIM). At this time, it is preferable that the "expiration date" indicated at a different position from the "identification number" is also read, along with the numbers of the "identification number".

[0136] Furthermore, the aiming position at which the worker shines the AIM light 40 of the optical information reader 10 may differ from the position of the symbol 20 that indicates the symbol information necessary for the work performed by the worker. For example, in an environment where workers are always instructed to shine the AIM light 40 on the barcode 23, the symbol information actually needed for the work may be the "expiration date," which is not encoded in the barcode 23.

[0137] In this case, it is preferable that the optical information reader 10 reads the "expiration date" which is located at a different position from the barcode 23, while the operator is shining the AIM light 40 onto the barcode 23. Hereinafter, the position of the symbol 20 to be read that is located at a different position from the position where the AIM light 40 is shining (targeting position) may be referred to as the reading target position 22.

[0138] It is preferable that the optical information reading device 10 is capable of reading symbol information at a reading target position 22, which is a position different from the aiming position. The reading target position 22 is a position identified based on the aiming position and reading target setting information generated according to the user's specifications. For example, the user specifies the relative position of the reading target position 22 with respect to the aiming position to which the AIM light 40 is irradiated. The optical information reading device 10 can identify the reading target position 22 based on the specified relative position and the aiming position. Hereinafter, with respect to an object 11 to which at least one symbol 20 is attached, the process by which the optical information reading device 10 performs image processing on the aiming position or the reading target position 22 to read the symbol information of one or more symbols 20 may be referred to as relative reading processing. Relative reading processing is performed by the relative reading processing unit 55 (Figure 4) of the optical information reading device 10. Relative reading processing is particularly effective when performed on standardized documents where it is predetermined what kind of information is written and where on the object 11. Examples of standardized documents include forms such as delivery slips. A form is an office paper with multiple fields for filling in various types of information.

[0139] Furthermore, the positional relationship (relative position) between the aiming position and the reading target position 22 may differ depending on the type of object 11 being imaged. For example, Figure 21 shows a Type 2 object 11b with multiple symbols 20 attached. A Type 2 object 11b is, for example, a shipping slip used in warehouse management. The arrangement of the symbols 20 differs between a Type 1 object 11a and a Type 2 object 11b. Therefore, even if a Type 1 object 11a and a Type 2 object 11b represent information about the same product, the positional relationship between the aiming position and the reading target position 22 will differ between a Type 1 object 11a and a Type 2 object 11b.

[0140] For example, in the first type object 11a in Figure 20, the "expiration date" is shown to the upper right of the "identification number," and this position is the reading target position 22. However, in the second type object 11b in Figure 21, the "expiration date," which is the reading target position 22, is located to the upper left of the "identification number." The optical information reading device 10 of this embodiment can read symbol information at the reading target position 22 for both the first type object 11a and the second type object 11b by relative reading processing, provided that the reading target setting information is set appropriately in advance. The optical information reading device 10 can also read symbol information of symbols 20 located at or near the aiming position during relative reading processing.

[0141] Figure 22 is a flowchart showing the flow of the reading process when relative reading is included in the reading process of step S16 in Figure 6. Prior to the relative reading process, pre-configuration (step S10) is performed. As part of this pre-configuration, the reading target configuration described later is performed, and the reading target configuration information used in the relative reading process is set in advance. Furthermore, it is preferable to determine which of the multiple symbols 20 attached to the object 11 can be read based on conditions predetermined in the pre-configuration. For example, if the symbol 20 is a string, it is preferable to determine whether or not it is a symbol 20 to be read based on predetermined format patterns or likelihood conditions for string likelihood.

[0142] When the reading mode, which includes relative reading processing, is activated, the optical information reading device 10 first generates an operational image 64 (Figure 5) in step S60, which includes the symbol 20 attached to the object 11, as one of the input images generated by the imaging unit 31. At this time, the operator illuminates the object 11 with the aimer light 40 at the desired position (or aligns the virtual aimer displayed on the display unit 14 to the desired position) and performs imaging with the imaging unit 31.

[0143] Next, in step S61, the optical information reading device 10 uses the relative reading processing unit 55 (Figure 4) included in the information processing unit 53 of the control unit 50 to identify the reading target position 22 based on the targeting position in the operational image 64 (the position corresponding to the AIM light 40) and the pre-set reading target setting information.

[0144] The relative reading processing unit 55, having identified the reading target position 22, performs relative reading processing in step S62 to read the symbol information of at least one symbol 20 by performing image processing on the reading target position 22 in the operational image 64. Here, if the reading target setting information is appropriately set for both the first type object 11a and the second type object 11b, the symbol information of the symbol 20 at the appropriate reading target position 22 will be read for both the first type object 11a and the second type object 11b.

[0145] Figure 23 is a flowchart showing the flow of setting the reading target for relative reading processing. In environments where relative reading processing is required (for example, facilities where warehouse management is performed), the operator performs setting the reading target as part of the pre-configuration before capturing operational images in relative reading processing. For example, setting the reading target is performed by selecting the "Setting Reading Target" item from the setting menu screen displayed on the display unit 14.

[0146] When the reading target setting is executed, the reading target setting unit 58 (Figure 4) of the information processing unit 53 generates reading target setting information. First, in step S63, the reading target setting unit 58 accepts registration of the required number of settings. The required number of settings is the number of reading target setting information required for the work performed by the operator, and specifically, the number of types of objects 11 that are to be imaged in the work performed by the operator and the number of relative positions of the reading target positions 22 required for the objects 11 are registered.

[0147] It is preferable that multiple types of reading target setting information be generated depending on the number of types of objects 11 to be imaged. For example, if there are two types of objects 11, such as Type 1 object 11a and Type 2 object 11b, it is preferable that two types of reading target setting information be generated. The number of types of reading target setting information is registered in step S63 by the operator's operation.

[0148] Then, for each of the multiple types of object 11, the number of relative positions of the required reading target positions 22 is registered. If the object 11 has multiple symbols 20 attached to it, multiple symbols 20 may be the target of reading. The number of relative positions of the reading target positions 22 registered in step S63 corresponds to the number of symbols 20 to be read.

[0149] In step S63, format information and layout information for the symbols 20 to be read may also be set. For example, format information such as the format of the string to be read, the barcode standard type, and the 2D code standard type may be set, as well as layout information on how each symbol 20 will be arranged on the object 11. Note that the various required settings do not necessarily need to be registered all at once at the start of setting the read targets; they may be registered as the operator proceeds with the read target setting procedure.

[0150] Next, in step S64, the imaging unit 31 images an area within the imaging field of view and generates a setting image 63 (Figure 5) that includes the symbol 20 attached to the object 11. When the setting image 63 is generated, the operator (user) carrying the optical information reading device 10 irradiates the AIM light 40 onto a position on the object 11 that will serve as the reference position for the relative reading process. With the AIM light 40 irradiated onto the reference position, the operator operates the trigger key 18 to generate the setting image 63 that includes the symbol 20 attached to the object 11. The generated setting image 63 does not necessarily have to include the AIM light 40, but the optical information reading device 10 can calculate the irradiation position (reference position in the relative reading process) where the AIM light 40 was irradiated in the setting image 63.

[0151] If there are multiple types of objects 11, a setting image 63 corresponding to the type of object 11 captured is generated. For example, if there are two types of objects 11, a first-type object 11a and a second-type object 11b, when a first-type object 11a is captured, a first setting image 63a corresponding to the first-type object 11a is generated. Similarly, when a second-type object 11b is captured, a second setting image 63b corresponding to the second-type object 11b is generated. If there are three or more types of objects 11, it is desirable that even more types of setting images 63 be generated according to the types of objects 11.

[0152] After the setting image 63 is generated, the specification reception unit 57 (Figure 4) receives the user's specification for the setting image 63 in step S65 of Figure 23. The user's specification received in step S65 indicates the relative positions of the multiple symbols 20 to be read with respect to the aiming position. Specifically, the setting image 63 is displayed on the display unit 14 or the monitor of the host computer 19 (Figure 2), and the user specifies the position (user-specified position) of the symbol 20 to be read from the displayed setting image 63. The specification reception unit 57 determines the position in the setting image 63 specified by the user as the user-specified position. If there are multiple symbols 20 to be read, the user specifies the position of each of the multiple symbols 20 in the setting image 63.

[0153] When the setting image 63 is displayed on the display unit 14, the operator (user) carrying the optical information reading device 10 specifies the user-specified position via the operation unit 15. For example, a cursor may be displayed on the display unit 14 along with the setting image 63, and the operator may move the cursor using the directional keys included in the operation unit 15 and specify the cursor's position as the user-specified position by operating the trigger key 18. Alternatively, if the display unit 14 is a touch panel display integrated with the operation unit 15, the operator may specify the user-specified position by directly touching the location on the setting image 63 displayed on the display unit 14.

[0154] When the setting image 63 is displayed on the monitor of the host computer 19, the communication unit 16 (Figure 4) transmits the setting image 63 to the host computer 19. The user operating the host computer 19 can then use a pointing device such as a mouse to specify a location on the setting image 63 displayed on the monitor as the user-specified location. If the monitor of the host computer 19 is a touch panel display, the user may specify the user-specified location by directly touching a location on the setting image 63 displayed on the monitor. The specification reception unit 57 receives the information of the specified user-specified location via the communication unit 16.

[0155] Then, in step S66, the reading target setting unit 58 generates reading target setting information. The reading target setting information includes the relative positional relationship between the position of the aiming position or the position of the symbol 20 adjacent to the aiming position in the setting image 63 and the user-specified position.

[0156] The reading target setting unit 58 generates reading target setting information 76 (Figure 5) based on the relative positional relationship between the user-specified position on the setting image 63 received by the designation reception unit 57 in step S65 and the aiming position of the AIM light 40 when the setting image 63 was captured. Here, the relative positional relationship between the user-specified position and the aiming position is included in the reading target setting information. If there are multiple types of objects 11 (e.g., first type object 11a, second type object 11b), reading target setting information 76 (e.g., first reading target setting information 76a, second reading target setting information 76b) corresponding to the type of object 11 captured is generated.

[0157] The targeting position onto which the AIM light 40 is emitted only needs to be near the symbol 20, and the targeting position does not need to precisely coincide with the position of the symbol 20. If the reading target setting unit 58 does not have a symbol 20 at the targeting position, it searches for the symbol 20 adjacent to the targeting position (the closest symbol 20 to the targeting position) within the setting image 63. The position of the symbol 20 adjacent to the targeting position is then treated as the reference position in the relative reading process instead of the targeting position. In the following explanation of the relative reading process, the targeting position and the position of the symbol 20 adjacent to the targeting position will be collectively referred to as the "reference position". The reading target setting information generated in step S66 of Figure 23 includes the relative positional relationship between the reference position and the user-specified position.

[0158] The relative reading processing unit 55 performs relative reading processing using the reading target setting information generated as described above. That is, the relative reading processing unit 55 identifies the reading target position 22 in the operational image 64 based on the relative positional relationship between the reference position and the user-specified position in the setting image 63. Specifically, in step S61 of Figure 22, the relative reading processing unit 55 first uses the position of the aiming position of the AIM light 40 or the position of the symbol 20 adjacent to the aiming position in the operational image 64 as the reference position. Then, the relative reading processing unit 55 identifies the position in the operational image 64 as the reading target position 22, where the positional relationship relative to the reference position corresponds to the user-specified position in the setting image 63.

[0159] If there are multiple user-specified positions, multiple reading target positions 22 are identified according to the number of user-specified positions. Also, if there are multiple types of reading target setting information, the type of reading target setting information corresponding to the type of imaged object 11 is used. For example, for a first-type object 11a, the first reading target setting information 76a is used. The type of reading target setting information to be used may be specified by the operator carrying the optical information reading device 10 or the user of the host computer 19, or it may be automatically selected by the optical information reading device 10. For example, if the layout information of the symbols 20 to be read is set for each type of object 11, the optical information reading device 10 analyzes the layout of the symbols 20 included in the operational image 64 and identifies the type of object 11 that matches that layout. The reading target setting information corresponding to the identified type of object 11 is then used for relative reading processing.

[0160] The following describes a specific example of setting the reading target, with reference to the drawings. Figure 24 shows the reading target setting screen 74 for setting the reading target. The reading target setting screen 74 in Figure 24 is titled "Document Reading Settings". "Document Reading Settings" means setting up the settings related to reading documents. When the operator performs an operation to start the reading target setting in the pre-settings, the reading target setting screen 74 is displayed on the display unit 14. For example, the reading target setting can be started when the operator selects an icon for starting the reading target setting from the setting menu screen displayed on the display unit 14. When the reading target setting is started, the screen generation unit 52 (Figure 4) of the optical information reading device 10 generates screen data for the reading target setting screen 74, and this screen data is sent to the output unit 12, so that the reading target setting screen 74 is displayed on the display unit 14.

[0161] The reading target setting screen 74 in Figure 24 includes an image display area 71 where the image captured by the imaging unit 31 is displayed, a message area 72 where a message to the operator is displayed, and an operation panel area 73 where operator operations are accepted.

[0162] When the setting of the reading target is started, the image display area 71 first displays a real-time image of the object 11 (in this case, a first-class object 11a) that is within the imaging field of view of the imaging unit 31. The real-time image is an image that is currently being captured by the imaging unit 31 and represents the state within the imaging field of view at that moment. At this time, it is preferable that the image display area 71 includes a real-time mark 71a (in this case, the string "LIVE") to indicate that it is a real-time image.

[0163] The message area 72 displays a message prompting the operator to align the aiming position of the AIM light 40 to the position that they wish to use as the reference position 75 for relative reading processing on the first type object 11a. In Figure 24, the message "Align the first element to the center and perform the scan." is displayed. This message prompts the operator to align the aiming position (the center of the AIM light 40) near the symbol 20 (the first element) located at the reference position 75. The AIM light 40 displayed in the image display area 71 is, but is not limited to, the AIM light irradiated by the AIM light irradiation unit 32 and reflected by the object 11. In other words, the display unit 14 may superimpose a virtual AIM at the position in the image display area 71 where the AIM light irradiated by the AIM light irradiation unit 32 and reflected by the object 11 is visible. Also, if the optical information reading device 10 is not equipped with an AIM light irradiation unit 32, a virtual AIM is superimposed in the center of the image display area 71. Preferably, the image display area 71 displays the entire imaging field of view of the imaging unit 31, that is, the range that will be imaged by the imaging unit 31. When the AIM light 40 (or virtual AIM) is illuminated (or displayed) with the center of the imaging field of view as the targeting position, the AIM light 40 (or virtual AIM) is displayed in the center of the image display area 71.

[0164] The control panel area 73 contains a back button 73a, a cancel button 73b, and a scan button 73c. ​​When the back button 73a is pressed, the display of the read target setting screen 74 returns to the previous work screen (the settings menu screen in Figure 24). When the cancel button 73b is pressed, the read target setting process is canceled, and the display of the read target setting screen 74 returns to the settings menu screen. When the scan button 73c is pressed, the setting image 63 is generated.

[0165] The operator adjusts the orientation (direction) of the optical information reading device 10 they are carrying to align the AIM light 40 with the reference position 75. In Figure 24, the reference position 75 is the position where the "identification number" information (digits) is written as symbol 20. In Figure 24, for explanatory purposes, the reference position 75 is surrounded by a dashed line, but the line surrounding the reference position 75 does not necessarily need to be displayed in the image display area 71.

[0166] The operator then operates the scan button 73c with the AIM light 40 aligned to the reference position 75. The imaging unit 31 then images the first type object 11a, and the first setting image 63a is generated. At this time, the position of the symbol 20 adjacent to the aiming position where the AIM light 40 was illuminating during imaging is treated as the reference position 75 in the first setting image 63a.

[0167] Furthermore, the operation for setting the target to be read may be performed by a user of the host computer 19 at a location separate from the optical information reading device 10. For example, when setting the target to be read is started, screen data for the target to be read screen 74, which includes real-time video of the first type object 11a captured by the imaging unit 31, is generated by the screen generation unit 52 (Figure 4) and transmitted to the host computer 19 via the communication unit 16 of the output unit 12. The target to be read screen 74, which includes real-time video of the first type object 11a, is then displayed on the monitor of the host computer 19.

[0168] The user of the host computer 19 instructs the operator carrying the optical information reader 10 to align the AIM light 40 with the reference position 75, and operates the scan button 73c when the AIM light 40 is aligned with the reference position 75. Alternatively, the reference position 75 may be specified by the pointing device of the host computer 19.

[0169] When the scan button 73c is operated, the display of the reading target setting screen 74 transitions to a state that accepts a user-specified position 78 for the first type object 11a, as shown in Figure 25. In this state, the first setting image 63a captured by the imaging unit 31 is displayed in the image display area 71. This reading target setting screen 74 (setting screen), including the first setting image 63a, is generated by the screen generation unit 52 (Figure 4) of the optical information reading device 10.

[0170] In Figure 25, the message area 72 displays the message, "Please specify the second element." This message prompts the operator to specify the position where the second symbol 20b (the second element), which should be the target of the relative reading process, is located in the first setting image 63a. Also in Figure 25, the operation panel area 73 displays a confirm button 73d instead of a scan button 73c.

[0171] Then, the read target setting screen 74 in Figure 25, which includes the first setting image 63a, contains the results of relative reading processing for the first symbol 20a at the reference position 75 (the symbol 20 that is at the target position in the first setting image 63a, or the symbol 20 adjacent to the target position). When the first setting image 63a is captured, the relative reading processing unit 55 performs image processing on the first symbol 20a at the reference position 75 and reads the symbol information represented by the first symbol 20a.

[0172] The read target setting screen 74, which includes the first setting image 63a, contains a first symbol information field 75a. In Figure 25, the first symbol information field 75a is displayed between the image display area 71 and the message area 72. The first symbol information field 75a displays the result of the relative reading process for the first symbol 20a. Here, the numerical value "123456" is displayed in the first symbol information field 75a as the symbol information represented by the first symbol 20a. The operator carrying the optical information reading device 10, or the user of the host computer 19, can check whether the symbol information of the first symbol 20a, which is located at the position that should be the reference position 75, has been read correctly by checking the contents displayed in the first symbol information field 75a. This also allows the operator carrying the optical information reading device 10, or the user of the host computer 19, to check whether the reference position 75 has been specified correctly.

[0173] In Figure 25, a setting change button 73e is displayed next to the first symbol information field 75a. When the setting change button 73e is operated, the display on the display unit 14 (or the monitor of the host computer 19) transitions to a screen (not shown) for changing the setting information related to the setting to be read. If the symbol information of the first symbol 20a is not displayed correctly in the first symbol information field 75a, the operator should operate the setting change button 73e to change the setting information for reading the symbol information of the first symbol 20a.

[0174] For example, if the object to be read is a string, settings such as the string's format pattern, margin threshold (the size of the whitespace to distinguish it from other strings), and string contrast (the difference in brightness to distinguish the character portion from the background portion) should be changed. Also, if the object to be read is an encoded code (a code in which information is encoded, such as a barcode or a two-dimensional code), an appropriate code standard type (such as CODE128) should be selected to decode that code.

[0175] After confirming that the symbol information for the first symbol 20a is correctly displayed in the first symbol information field 75a, the operator then specifies the user-specified position 78. The specification reception unit 57 (Figure 4) of the optical information reading device 10 receives the operator's (user's) specification and determines the user-specified position 78 in the reading target setting screen 74 (setting screen).

[0176] The operator designates the position where the second symbol 20b, which they wish to be read relative to the setting image 63 (in this case, the first setting image 63a) displayed on the display unit 14, is located as the user-specified position 78. For example, a cursor 79 is displayed on the display unit 14, and the operator moves the cursor 79 using the directional keys included in the operation unit 15 to position it at the user-specified position 78. In this case, the cursor 79 is positioned at the location of the second symbol 20b, which indicates the "expiration date," located to the upper right of the reference position 75.

[0177] When the cursor 79 is positioned at the location of the second symbol 20b, and the operator operates the confirm button 73d, the designation reception unit 57 accepts the user's designation for the first setting image 63a and determines the position within the first setting image 63a specified by the user (in this case, the position of the second symbol 20b) as the user-specified position 78. In Figure 25, for illustrative purposes, the user-specified position 78 is surrounded by a dashed line, but the line surrounding the user-specified position 78 does not necessarily need to be displayed in the image display area 71.

[0178] Alternatively, the user-specified position 78 may be determined by the user of the host computer 19 specifying a position on the first setting image 63a displayed on the monitor of the host computer 19 using a pointing device. If the display unit 14 (or the monitor of the host computer 19) is a touch panel display, the operator (user) may touch the position on the first setting image 63a that will be the user-specified position 78.

[0179] Once the user-specified position 78 is determined, the display of the reading target setting screen 74 transitions to a state that displays a confirmation message for the specified position setting, as shown in Figure 26. In Figure 26, continuing from the state in Figure 25, the first setting image 63a captured by the imaging unit 31 is displayed in the image display area 71.

[0180] In Figure 26, the message "If there are no problems, please select OK" is displayed in the message area 72. This message prompts the operator to confirm whether there are any problems with the user-specified position 78.

[0181] The reading target setting screen 74 in Figure 26 includes the results of relative reading processing for the second symbol 20b located at the user-specified position 78 in the first setting image 63a. When the designation reception unit 57 determines the user-specified position 78, the relative reading processing unit 55 performs image processing on the second symbol 20b at the user-specified position 78 and reads the symbol information represented by the second symbol 20b.

[0182] The reading target setting screen 74 in Figure 26 includes a second symbol information field 78a that displays the result of the relative reading process for the second symbol 20b. In Figure 26, the second symbol information field 78a is displayed below the first symbol information field 75a. Here, the string "2025.11.19", representing the expiration date, is displayed in the second symbol information field 78a as the symbol information represented by the second symbol 20b. The operator carrying the optical information reading device 10, or the user of the host computer 19, can check whether the symbol information of the second symbol 20b located at the user-specified position 78 has been correctly read by checking the contents displayed in the second symbol information field 78a. This also allows the operator carrying the optical information reading device 10, or the user of the host computer 19, to check whether the user-specified position 78 has been correctly specified.

[0183] Next to the second symbol information field 78a, a setting change button 73e is displayed. If the symbol information for the second symbol 20b is not displayed correctly in the second symbol information field 78a, the operator (user) should use the setting change button 73e to change the setting information for reading the symbol information for the second symbol 20b.

[0184] After confirming that the symbol information for the second symbol 20b is correctly displayed in the second symbol information field 78a, the operator (user) confirms the user-specified position 78 by operating the confirm button 73d on the operation panel area 73.

[0185] Once the user-specified position 78 for the first setting image 63a is confirmed, the reading target setting unit 58 (Figure 4) of the optical information reading device 10 generates reading target setting information 76 (first reading target setting information 76a) for the first type object 11a and stores it in the storage unit 60. The reading target setting information 76 includes the relative positional relationship between the reference position 75 (the aiming position or the position of the first symbol 20a adjacent to the aiming position) and the user-specified position 78 in the setting image 63 (first setting image 63a). The positional relationship included in the reading target setting information 76 may be stored as absolute coordinates within the first setting image 63a, for example. For example, for an element within the first setting image 63a identified as a symbol 20 located at the reference position 75 or the user-specified position 78, the coordinates of the four corners (four points) of the rectangle identified as the range of the symbol 20 may be stored. Alternatively, the origin may be a position identified within the first setting image 63a, such as the center of the Aima light 40, and the relative coordinates of the four corners of each symbol 20 with respect to that origin may be stored.

[0186] The reading target setting information 76 may include, in addition to the relative positional relationship between the reference position 75 and the user-specified position 78, various information used in the relative reading process. For example, the reading target setting information 76 may include the format pattern of the symbols 20 that are subject to relative reading (potential reading targets) in the setting image 63, the number of reading targets (number of user-specified positions 78), the layout information of the symbols 20 (arrangement of user-specified positions 78), and the size (first size) of the symbols 20 (first symbol 20a and second symbol 20b) that are subject to relative reading in the setting image 63. The reading target setting unit 58 may, for example, measure the number of pixels occupied by the symbols 20 in the setting image 63 and set this as the first size. At this time, the distance from the optical information reading device 10 to the object 11 (first type object 11a) may be measured by the distance measuring unit 34, and the measurement result may be included in the reading target setting information 76.

[0187] Figure 27 shows the setting status of the reading target for the first type object 11a. When the reading target setting unit 58 generates the reading target setting information 76, the display of the reading target setting screen 74 transitions to a state that shows the setting status of the reading target. The reading target setting screen 74 in Figure 27 includes a preset selection area 81, a setting name display area 82, a setting status display area 83, and an operation panel area 73.

[0188] The preset selection area 81 is an area where the operator can select which of several presets related to the read target setting information 76 to apply. A preset is an existing setting that has been prepared in advance for the read target setting information 76. Multiple presets are available depending on the type of relative reading process performed by the operator. The content of the currently applied preset can be changed by the operator by setting the read target.

[0189] The operator selects a preset to apply to the relative reading process they are performing, or a preset to modify the content of. Four presets are shown here, and the "Setting 1" preset is selected. Figure 27 shows the state in which the read target setting information 76 included in the "Setting 1" preset has been modified by the read target setting.

[0190] The setting name display area 82 is an area that displays the names assigned to the reference position 75 and user-specified position 78, which are set by the reading target setting. In Figure 27, the initial names that the reading target setting unit 58 automatically assigns are "Reading Setting 1" to the reference position 75 and "Reading Setting 2" to the user-specified position 78. These names can be changed by the operator. Each name displayed in the setting name display area 82 is assigned a number (here, "1" and "2"). These numbers correspond to the numbers assigned to the reference position 75 and user-specified position 78 in the setting status display area 83.

[0191] The setting status display area 83 is an area that shows the setting status of the reading target set by the reading target setting. Figure 27 shows the first setting figure 83a which shows the setting status of the reading target for the first type object 11a. Specifically, the first setting figure 83a shows the relative positional relationship between the reference position 75 and the user-specified position 78.

[0192] The first setting shape 83a is represented by a rectangular frame, within which are a rectangle representing the reference position 75 and a rectangle representing the user-specified position 78. In Figure 27, the reference position 75 is located in the center of the first setting shape 83a, and the user-specified position 78 is located relatively to the upper right of the reference position 75. The rectangle representing the reference position 75 and the rectangle representing the user-specified position 78 are numbered "1" and "2," respectively. These numbers indicate that each rectangle within the first setting shape 83a corresponds to a name displayed in the setting name display area 82.

[0193] The number "1)" is shown to the left of the first setting figure 83a. This number indicates that the first setting figure 83a corresponds to the first reading target setting information 76a based on the first setting image 63a obtained by imaging the first type object 11a. The first reading target setting information 76a is used when relative reading processing is performed on the first type object 11a.

[0194] The control panel area 73 includes an add button 73f and a delete button 73g. When the add button 73f is operated, a setting state for the reading target is added. Here, the process of adding the setting for the reading target related to the second type of object 11b is started. When the delete button 73g is operated, the reading target setting information 76 is deleted. If there are multiple reading target setting information entries 76, the operator can delete the reading target setting information 76 corresponding to a specific type of object 11 by selecting the shape corresponding to the reading target setting information 76 to be deleted from the shapes displayed in the setting state display area 83 and then operating the delete button 73g.

[0195] In the upper left corner of the read target setting screen 74 in Figure 27, a setting screen exit icon 89 ("<" mark) is displayed. When this setting screen exit icon 89 is operated, the operation on the read target setting screen 74 is temporarily interrupted, and the display on the display unit 14 returns to the state before the read target setting screen 74 was displayed (for example, the setting menu screen).

[0196] In Figure 27, an extended menu icon 84 (a mark with three dots in a row) is displayed in the upper right corner of the read target settings screen 74. When this extended menu icon 84 is operated, extended menu items 84m are displayed. The extended menu items 84m may include, for example, a user application association button 84a for associating the read target settings information 76 with a user application 90, a setting name change button 84b for changing the name displayed in the setting name display area 82, and a setting clear button 84c for erasing (clearing) the various settings set on the read target settings screen 74. For illustrative purposes, in Figure 27, the extended menu items 84m are shown outside the read target settings screen 74, but in reality, the extended menu items 84m should be displayed inside the read target settings screen 74, overlaid with the preset selection area 81, the setting name display area 82, etc.

[0197] If there are multiple types of objects 11, the operator operates the add button 73f to start the process of adding a reading target setting for an object 11 of a different type from the first type object 11a (second type object 11b). The process of adding a reading target setting for the second type object 11b is the same as the process of setting a reading target for the first type object 11a, except that the imaging target is the second type object 11b.

[0198] In other words, the operator aligns the EIM light 40 with the position of the first symbol 20a of the second type object 11b and takes an image of the second setting image 63b (same as in Figure 24). The operator then confirms that the symbol information of the first symbol 20a at the reference position 75 of the second setting image 63b has been read correctly, and then specifies the position of the second symbol 20b as the user-specified position 78 (same as in Figure 25). Then, a confirmation message for the specified position setting of the second type object 11b (element 2), "If there are no problems, please select OK," is displayed in the message area 72 of the reading target setting screen 74.

[0199] Figure 28 shows the read target setting screen 74 with a confirmation message for the specified position setting for the second type object 11b displayed. In the read target setting for the second type object 11b, the second setting image 63b is displayed in the image display area 71. In the second setting image 63b in Figure 28, the upper left position of the central reference position 75 is specified as the user-specified position 78.

[0200] Once the operator confirms that the symbol information for the second symbol 20b at the user-specified position 78 is correctly displayed in the second symbol information field 78a, they confirm the user-specified position 78 for the second setting image 63b by operating the confirm button 73d.

[0201] Once the user-specified position 78 for the second setting image 63b is confirmed, the reading target setting unit 58 generates second reading target setting information 76b for the second type object 11b and stores it in the storage unit 60. Subsequently, the display of the reading target setting screen 74 transitions to a state indicating the setting status of the reading target.

[0202] The reading target setting screen 74 in Figure 29 shows the setting status of the reading targets for Type 1 object 11a and Type 2 object 11b. In this state, the reading target setting screen 74 shows, in addition to the display in Figure 27, a second setting figure 83b indicating the setting status of the reading target for Type 2 object 11b is shown in the setting status display area 83. The second setting figure 83b shows the relative positional relationship between the reference position 75 set in the second setting screen 74b and the user-specified position 78. In Figure 29, it is shown that the user-specified position 78 is located relatively to the upper left with respect to the central reference position 75.

[0203] The number "2)" is shown to the left of the second setting figure 83b. This number indicates that the second setting figure 83b corresponds to the second reading target setting information 76b, which is based on the second setting image 63b obtained by imaging the second type object 11b. The second reading target setting information 76b is used when relative reading processing is performed on the second type object 11b.

[0204] The reading target setting unit 58 may also include in the reading target setting information the relative positional relationship between the user-specified area 78z, which includes the user-specified position 78, and the reference position 75 (the position of the aiming position or a symbol 20 adjacent to the aiming position) in the setting image 63. The user-specified area 78z is an area that includes the user-specified position 78 and is wider than the user-specified position 78, as shown by the dashed line (double-dotted line) in Figure 29. For example, if the user-specified position 78 is often located to the left of the reference position 75 in multiple objects 11, it is preferable to set a user-specified area 78z that includes the user-specified position 78 instead of setting a user-specified position 78 for each of the multiple types of objects 11.

[0205] For example, when a setting image 63 (such as the first setting image 63a in Figure 25 or the second setting image 63b in Figure 28) is displayed in the image display area 71, the user-specified area 78z can be specified by dragging a pointing device (or flicking a touch panel). Alternatively, as shown in Figures 27 and 29, when the setting status of the object to be read is displayed, it may be possible to enlarge the range of the user-specified position 78 using a pointing device and reset it as the user-specified area 78z. In addition, the user-specified area 78z may be specified by (1) specifying only directions such as "right," "left," "up," or "down" with respect to the reference position 75, (2) specifying a range within a predetermined distance from the reference position 75, that is, a circular range centered on the reference position 75, as the user-specified area 78z, or (3) specifying a range of a predetermined angle with respect to the horizontal axis passing through the reference position 75 as the user-specified area. Furthermore, as an alternative method for setting the elements to be read, it is also possible to set the elements to be read by: (4) specifying the positional relationship of only some of the multiple elements (symbols) to be read with respect to the reference position 75, while leaving the positions of the remaining elements arbitrary (they can be anywhere); (5) specifying the elements to be read in order of their relative distance from the reference position 75; or (6) treating the user-specified position 78 as a new reference position and storing the position of the third element to be read as its relative position from this new reference position. The elements to be read setting unit 58 should generate the elements to be read setting information so that the relative reading processing unit 55 can read information from the elements to be read set in the manner described above.

[0206] When an operator handles an object 11, such as in warehouse management, they capture an operational image 64 by imaging the object 11. At this time, the operator irradiates the object 11 with EMA light 40 to specify a reference position 75. Based on the reading target setting information 76 (first reading target setting information 76a, second reading target setting information 76b) set as described above, the relative reading processing unit 55 identifies the reading target position 22 in the operational image 64. The relative reading processing unit 55 then performs image processing on the reading target position 22 to read the symbol information represented by the symbol 20 located at the reading target position 22, which is different from the reference position 75. At this time, the symbol information of the symbol 20 located at the reference position 75 may also be read in addition to the symbol 20 at the reading target position 22.

[0207] Since the operator specifies the reference position 75 using the AIM optics 40 both when capturing the setting image 63 and when capturing the operation image 64, even if the setting image 63 and the operation image 64 are captured under different imaging conditions, the relative reading processing unit 55 can identify the reading target position 22 corresponding to the user-specified position 78 without any problems by checking the relative position from the reference position 75 in the operation image 64. This enables the optical information reading device 10 to stably read symbol information.

[0208] The reading target setting unit 58 may also measure the size (second size) of the symbol 20 that was the target of relative reading in the operational image 64. If the reading target setting information includes the first size of the symbol 20 that was the target of relative reading in the setting image 63, the reading target setting unit 58 may identify the position in the operational image 64 corresponding to the user-specified position 78 based on the comparison result between the first size and the second size.

[0209] Alternatively, the reading target setting unit 58 may measure the distance to the target object 11 using the distance measuring unit 34 when generating the operational image 64, and identify the position in the operational image 64 corresponding to the user-specified position 78 based on the comparison result of the measured distance and the distance measured when generating the setting image 63.

[0210] For example, the reading target setting unit 58 calculates the actual dimensions of the symbol 20 to be read by comparing the distance between the imaging unit 31 and the object 11 when the setting image 63 is captured with a first size. Then, the reading target setting unit 58 compares the distance between the imaging unit 31 and the object 11 when the operation image 64 is captured with a second size, and identifies the position of the symbol 20 whose dimensions match the actual dimensions calculated from the setting image 63 as the position corresponding to the user-specified position 78. The reading target setting unit 58 reads the symbol information of the symbol 20 located at the position corresponding to the user-specified position 78.

[0211] The symbol information read in this way is used for various tasks performed by the worker, such as warehouse management. For example, the read symbol information may be stored in an input field 91 of a user application 90 (Figure 5) used by the worker. By associating the read target setting information 76 with the user application 90 in advance, it is determined which symbol information will be stored in which input field 91.

[0212] The association between the read target setting information 76 and the user application 90 is initiated, for example, by the operator operating the user application association button 84a from the extended menu item 84m in Figure 29. When the user application association button 84a is operated, the display of the read target setting screen 74 transitions to a screen that displays a message prompting the user application 90 to be launched, as shown in Figure 30.

[0213] The reading target setting screen 74 in Figure 30 displays the message, "Please launch the application to associate with the document and perform the reading settings." This message prompts the user to launch a user application 90 that has an input field 91 where the symbol information of the symbol 20 attached to the object to be imaged (in this case, the document) should be stored. When the operator operates the association button 73h displayed on this screen, the display on the display unit 14 transitions to the application selection screen 85, as shown in Figure 31.

[0214] The application selection screen 85 displays icons for launching various applications used by the operator. Figure 31 shows several icons, including "Start Read Mode," "Register Format," "Set Read Target," and "User Application." The operator uses the cursor 79 to select the icon for the desired user application 90.

[0215] When transitioning from the reading target setting screen 74 to the application selection screen 85, an input field selection panel 86 is displayed in a portion of the application selection screen 85 (in this case, the lower part). The input field selection panel 86 includes a select button 86a, a complete button 86b, and a cancel button 86c. The operator confirms the selection of the user application 90 by operating the select button 86a while the icon of the desired user application 90 is selected with the cursor 79. If the display unit 14 is a touch panel display, the user application 90 may also be selected by the operator touching the icon of the user application 90.

[0216] When the Complete button 86b is pressed, the settings information for the input fields 91 selected up to that point is included in the Read Target Settings Information 76 and stored, completing the association with the user application 90. When the Cancel button 86c is pressed, the settings information for the input fields 91 selected up to that point is canceled. When either the Complete button 86b or the Cancel button 86c is pressed, the display on the display unit 14 returns to the Read Target Settings screen 74 as it was before the User Application Association button 84a was pressed.

[0217] When the icon for user application 90 is selected on the application selection screen 85, the display on the display unit 14 transitions to the screen of user application 90. Figure 32 shows the screen of user application 90, which has multiple input fields 91.

[0218] The user application 90 only needs to have one or more input fields 91, including a first input field 91a, but in this case, multiple input fields 91 are provided. In Figure 32, the first input field 91a corresponding to "Expiration Date" is displayed on the screen of the user application 90. The user application 90 in Figure 32 also has a second input field 91b, which is different from the first input field 91a. The second input field 91b in Figure 32 corresponds to "Identification Number". The user application 90 may also have many more input fields 91, such as a third input field 91c corresponding to "Code".

[0219] The operator uses the cursor 79 to select the first input field 91a where the symbol information (reading result of relative reading process) of the symbol 20 at the reading target position 22 should be stored, and then operates the selection button 86a (or touches the first input field 91a).

[0220] When the first input field 91a is selected, the display on the display unit 14 transitions to a state that includes the read target setting screen 74 shown in Figure 33. In this state, the read target setting screen 74 displays a read target determination panel 87 instead of the operation panel area 73, compared to the state in Figure 29. The read target determination panel 87 includes a select button 87a and a cancel button 87b. Additionally, a radio button 87c is added to the setting name display area 82 for selectively selecting either the reference position 75 or the user-specified position 78. When the cancel button 87b is operated, the setting information related to the input field 91 selected up to that point is canceled, and the display on the display unit 14 returns to the read target setting screen 74 before the user application association button 84a was operated.

[0221] The operator selects the name of the reference position 75 or user-specified position 78 corresponding to the symbol information to be stored in the first input field 91a selected in Figure 32 using the radio button 87c. In Figure 33, the name of the user-specified position 78 corresponding to "Expiration Date," "Reading Setting 2," is selected using the radio button 87c. When the operator operates the confirm button 87a in this state, the user-specified position 78 is associated with the first input field 91a. Specifically, the reading target setting unit 58 stores the reading target setting information 76, which further includes information associating the result of the relative reading process for the user-specified position 78 with the first input field 91a, in the storage unit 60.

[0222] By associating the user-specified position 78 with the first input field 91a, the symbol information for the "expiration date" of the reading target position 22 corresponding to the user-specified position 78 will be stored in the first input field 91a, regardless of whether the object being imaged 11 is a first-class object 11a or a second-class object 11b.

[0223] When the confirmation button 87a is pressed, the display on the display unit 14 returns to the application selection screen 85 in Figure 31. If there are other input fields 91 besides the first input field 91a that should be associated with the reference position 75 or the user-specified position 78, the operator continues the association process. For example, it is desirable to associate the second input field 91b with the reference position 75. Once the second input field 91b is associated with the reference position 75, the reading target setting information 76 will further include information that associates the result of the relative reading process with respect to the reference position 75 (the aiming position or the position of the symbol 20 adjacent to the aiming position) with the second input field 91b.

[0224] Once all associations between the reference position 75 or user-specified position 78 and the input field 91 have been completed, the operator should complete the association by pressing the "Complete" button 86b on the application selection screen 85.

[0225] Once the reference position 75 or user-specified position 78 is associated with the input field 91 in the manner described above, when relative reading is performed on the object 11, the symbol information to be stored in the input field 91 is automatically read and stored. Figure 34 shows the correspondence between the symbols 20 on the object 11 (Type 1 object 11a, Type 2 object 11b) and the input field 91 of the user application 90.

[0226] On the left side of Figure 34, an operational image 64 of a Type 1 object 11a is shown. On the right side of Figure 34, an operational image 64 of a Type 2 object 11b is shown. In the center of Figure 34, a screen of a user application 90 with multiple input fields 91 is shown. Note that the actual operational image 64 does not include the AIM light 40, and the dashed lines (double-dot lines) surrounding the reference position 75 and the reading target position 22 are not displayed, but they are shown in Figure 34 for illustrative purposes.

[0227] Furthermore, it is assumed that the operator has previously associated the user-specified position 78 (reading target position 22) with the first input field 91a ("Expiration Date") and the reference position 75 with the second input field 91b ("Identification Number"). Also, in Figure 34, it is assumed that the relative position of the barcode 23 with respect to the reference position 75 is associated with the third input field 91c ("Code").

[0228] When an operator performs an operation using a Type 1 object 11a (e.g., an inbound slip) (e.g., inbound registration in warehouse management), the operator illuminates the Type 1 object 11a with AIM light 40 so that the "identification number" is at the reference position 75, thereby capturing an operational image 64. At this time, it is preferable that the operator specifies that the object to be captured is a Type 1 object 11a (using the first reading target setting information 76a), but the optical information reading device 10 may also determine this automatically.

[0229] The relative reading processing unit 55 identifies the reading target position 22 based on the reference position 75 (the position of the symbol 20 adjacent to the aiming position or the aiming position) and the first reading target setting information 76a, and stores the result of the relative reading process for the reading target position 22 in the first input field 91a. As a result, the symbol information of the "expiration date" (in this case, the date "2025.11.19") at a position different from the "identification number" irradiated by the user with AIM light 40 is read and automatically stored in the first input field 91a.

[0230] At this time, the result of the relative reading process with respect to the reference position 75 (the position of the symbol 20 adjacent to the aiming position) (symbol information of the "identification number," in this case the numbers "123456") may be automatically stored in the second input field 91b. Also, the result of the relative reading process with respect to the barcode 23 (the information represented by the barcode 23, in this case the code "********") may be automatically stored in the third input field 91c.

[0231] Similarly, when an operator performs work using a Type 2 object 11b (e.g., a shipping slip) (e.g., shipping registration in warehouse management), they irradiate the Type 2 object 11b with AIM light 40 so that the "identification number" is at the reference position 75, and capture an operational image 64. Then, based on the second reading target setting information 76b, the relative reading processing unit 55 performs relative reading on the reading target position 22, the reference position 75, and the barcode 23 on the Type 2 object 11b, and stores the results in the first input field 91a, the second input field 91b, and the third input field 91c, respectively.

[0232] In this way, by aligning the AIM light 40 with the reference position 75 and capturing the operational image 64, the operator can obtain symbol information not only from the "identification number" at the reference position 75, but also from positions different from the reference position 75. Furthermore, since the obtained symbol information is automatically stored in each of the input fields 91, the effort required for the operator to operate the user application 90 and input into the input fields 91 is reduced.

[0233] Furthermore, if reading target setting information 76 for multiple types of objects 11 is set in advance, the operator can obtain the desired symbol information for multiple types of objects 11 with different arrangements (layouts) of symbols 20 using a common operation. That is, for any type of object 11, symbol information can be obtained from the reading target position 22, which is located at a different position depending on the type of object 11, by using a common operation of aligning the AIM light 40 with the reference position 75 and capturing an operational image 64. Therefore, symbol information can be read using a common operation regardless of the type of object 11, reducing the workload of the operator.

[0234] Next, we will describe another example of the reading process in step S16 of Figure 6. As shown in Figure 35, the object 11 may have multiple symbols 20 attached as a matrix aligned horizontally (rows) and vertically (columns). These aligned symbols 20 may represent structured data.

[0235] For example, in Figure 35, the multiple symbols 20 arranged in the row direction represent a set of data (record) having multiple attributes (fields, data items) such as "product name" and "unit price". The multiple symbols 20 arranged in the column direction represent the attributes of each record. The object 11 in Figure 35 is, for example, an inventory sheet in warehouse management, and each record corresponds to a product in the warehouse. Here, the attributes of each record represent "No." (the number assigned to the product), "product name", "quantity", "unit price", and "price". The format in which these attributes are attached to the object 11 is defined, such as numerical values, strings, or encoded codes (barcodes, etc.). It is preferable that the symbolic information represented by these multiple symbols 20 be output as structured data.

[0236] Here, the object 11 in Figure 35 has grid lines that divide each symbol 20 vertically and horizontally. However, the optical information reading device 10 of this embodiment uses an aima light 40 (or virtual aima) instead of grid lines to identify the arrangement of the symbols 20. When the operator reads the symbols 20 arranged as a matrix, they should irradiate the aima light 40 so that the first direction 40H extending from the first aima light 40a is aligned with the row direction of the matrix (or adjust the position of the virtual aima displayed on the display unit 14). If the first direction 40H is aligned with the row direction of the matrix, then the second direction 40V intersecting the first direction 40H will be aligned with the column direction of the matrix.

[0237] The matrix processing unit 56 (Figure 4) of the optical information reading device 10 performs matrix processing to identify the row and column in the matrix where each of the multiple symbols 20 is located, based on the direction in which the AIM light 40 extends. By performing matrix processing based on the direction in which the AIM light 40 extends, the matrix processing unit 56 can identify the row and column of each symbol 20 with limited computing power, without placing an excessive computational load on a processor of a size that can be installed in a portable optical information reading device 10. Furthermore, by performing matrix processing based on the direction in which the AIM light 40 extends, the optical information reading device 10 can acquire structured matrix data from an object 11 that does not have grid lines.

[0238] The flowchart in Figure 36 shows the flow of matrix processing. First, in step S70, the matrix processing unit 56 performs image processing on the input image 62 generated by the imaging unit 31 imaging the object 11, thereby detecting multiple symbols 20 contained in the input image 62. The detection of symbols 20 may be performed using, for example, a machine learning model 61 (Figure 5). If the symbols 20 are strings, then character detection is performed first, and then multiple characters that are estimated to be highly related are concatenated as a string.

[0239] When multiple symbols 20 contained in the input image 62 are detected, the matrix processing unit 56, in step S71, identifies the row and column in the matrix for each of the multiple symbols 20 based on the orientation of the AIM light 40.

[0240] Then, in step S72, the output unit 12 (Figure 4) of the optical information reading device 10 outputs the symbol information represented by the multiple symbols 20 as structured matrix data, based on the rows and columns identified by the matrix processing unit 56. For example, the structured data may be displayed as an image on the display unit 14 of the output unit 12, transmitted to the host computer 19 via the communication unit 16, or stored in the storage unit 60. The output structured data can be any format that can describe multiple records together, such as a CSV (Character Separated Values) file, an XML (Extensible Markup Language) file, a tabular data file, or a JSON (JavaScript Object Notation) file.

[0241] As a concrete example of matrix processing, let's explain the case where symbol 20 is a string of characters. Figure 37 shows an example of an input image 62 obtained when an object 11 with multiple strings of characters attached as symbol 20 is photographed at an angle. As shown in Figure 37, depending on the orientation of the object 11 or the orientation of the optical information reading device 10, the object 11 may be photographed at an angle relative to the outer frame of the input image 62. In this case, even if the object 11 is a rectangular piece of paper, the four sides of that rectangle will be at an angle relative to the outer frame (rectangle) of the input image 62. In this case, the direction of the arrangement of each character in the string of characters, and the row and column directions of the matrix to which symbol 20 belongs, will also be at an angle relative to the outer frame of the input image 62. In reality, operators try to align the orientation of the object 11 with the outer frame of the input image 62 as much as possible when photographing, so the object 11 is rarely tilted significantly, but for illustrative purposes, Figure 37 shows the object 11 tilted significantly relative to the outer frame of the input image 62.

[0242] Thus, if the object 11 is tilted relative to the outer frame of the input image 62, the horizontal and vertical directions in the input image 62 do not coincide with the row and column directions of the matrix formed by the multiple symbols 20. However, the optical information reading device 10 of this embodiment can identify the row and column in which each of the symbols 20 is located in the matrix by considering the first direction 40H to which the AIM light 40 extends.

[0243] When imaging the object 11, the operator irradiates the object with the aima light 40 such that the first direction 40H of the first aima light 40a (or virtual aima) of the aima light 40 is aligned as closely as possible with the direction of the characters and the row direction of the matrix (adjusting the position of the virtual aima). As shown in Figure 1, the aima light irradiation unit 32 is included in the same imaging module 13 as the imaging unit 31, so the orientation of the aima light irradiation unit 32 relative to the object 11 and the orientation of the imaging unit 31 are almost the same. Therefore, in practice, if the first direction 40H is aligned with the row direction of the matrix, the orientation of the outer frame of the input image 62 generated by the imaging unit 31 will also be close to the row and column direction of the matrix. Therefore, the first direction 40H is rarely tilted significantly with respect to the outer frame of the input image 62, but for illustrative purposes, Figure 37 shows a state in which the first direction 40H is tilted with respect to the outer frame of the input image 62.

[0244] The matrix processing unit 56 first performs image processing on the input image 62 to detect multiple symbols 20 contained in the input image 62. Specifically, the matrix processing unit 56 can detect multiple symbols 20 by inputting the input image 62 into the machine learning model 61. In Figure 37, the symbols 20 are multiple strings, and each string consists of multiple characters 25. For example, the matrix processing unit 56 inputs the input image 62 into the machine learning model 61 to obtain information about the characters 25 output by the machine learning model 61. The information about the characters 25 output by the machine learning model 61 includes the coordinates of the characters 25 in the input image 62. The machine learning model 61 can output information about the characters 25 even if the characters 25 are tilted in the input image 62.

[0245] The matrix processing unit 56 determines the positional relationship of multiple characters 25 based on the coordinates of the characters 25. Then, based on the determined positional relationship, the matrix processing unit 56 selects at least two characters to be used to determine whether or not to perform character concatenation to form a string. For example, the matrix processing unit 56 may calculate the distance between characters 25. The coordinates of the characters 25 output by the machine learning model 61 are represented, for example, as data representing the coordinates of the four corners (four points) of the rectangle surrounding the characters 25. The matrix processing unit 56 calculates the center coordinates of each character 25 (the coordinates of the center of the rectangle surrounding the characters 25) based on the coordinates of the four corners of the characters 25. Then, the matrix processing unit 56 calculates the distance between the center coordinates of each character 25 as the distance between the characters 25. The distance between characters 25 can be calculated even if the characters 25 are tilted. Characters 25 with a short distance between them are likely to belong to the same string.

[0246] Figure 37 illustrates, as a specific example, the distance ΔD between two letters 25 (the alphabet "h") located in the upper left of object 11, and the distance ΔD between two letters 25 (the alphabet "c") located in the lower right of object 11. This distance ΔD is calculated as the distance between the centers 25a of the two letters 25.

[0247] The matrix processing unit 56 selects two characters with a sufficiently small distance ΔD (e.g., less than 10 pixels) to be considered for character concatenation. When deciding whether or not to concatenate characters, it is desirable to consider not only the distance between characters 25 (the positions of multiple characters), but also the relationship between characters 25, such as their type (alphabet, kanji, etc.) and size. Multiple characters 25 that are estimated to be highly related based on their positions and sizes are grouped together as a string. For the purposes of the following explanation, the string consisting of multiple characters 25 grouped (concatenated) by the matrix processing unit 56 will be called a concatenated string 26. The matrix processing unit 56 obtains multiple concatenated strings by concatenating multiple characters based on their positions and sizes. In Figure 37, characters 25 with small distances ΔD are concatenated to form seven concatenated strings 26 (seven concatenated strings 26 are obtained by the matrix processing unit 56).

[0248] Furthermore, when the matrix processing unit 56 determines the positional relationship of multiple characters, the first direction 40H of the AIM light 40 may be taken into consideration. For example, the matrix processing unit 56 determines the X direction (indicated by the symbol "X" in Figure 37) corresponding to the first direction 40H in the input image 62, and the Y direction (indicated by the symbol "Y" in Figure 37) perpendicular to the X direction. The X direction is typically parallel to the first direction 40H, but the X direction may be a direction at an angle to the first direction 40H based on the orientation of the optical information reading device 10, the inclination of the object 11, etc.

[0249] The matrix processing unit 56 then sorts the multiple characters 25 based on their coordinates, in the order they are arranged in the X direction and in the Y direction. For the sorted characters 25, the matrix processing unit 56 selects two characters that are aligned along either the X or Y direction and determines whether or not to concatenate them. By sorting the multiple characters 25 and then selecting which characters to concatenate, it becomes unnecessary to examine all possible combinations of characters 25, thereby reducing the processing time and processing load on the processor installed in the optical information reading device 10.

[0250] The matrix processing unit 56 may also calculate the orientation of at least one of the multiple symbols 20 (here it is a string, but it may also be an encoded code, etc.) and correct its orientation. As an example, Figure 37 shows a state in which the orientation of one of the symbols 20, the concatenated string 26 in the lower left (a string formed by concatenating three letters "n"), forms an angle with respect to the first direction 40H. The orientation of the symbol 20 refers to its orientation within the input image 62. Specifically, if the symbol 20 is a concatenated string 26, the direction of the arrangement of the characters 25 becomes the orientation of the symbol 20. When the symbol 20 is extracted from the input image 62, if the coordinates of the four corners (four points) of the rectangle identified as the range of the symbol 20 are obtained, the direction of the sides of that rectangle becomes the orientation of the symbol 20. Furthermore, when the matrix processing unit 56 detects a symbol 20, if it can detect that the symbol 20 is tilted relative to the outer frame of the input image 62, the direction of that tilt becomes the orientation of the symbol 20.

[0251] Here, as an example, we will explain the case in which the matrix processing unit 56 calculates the orientation of the concatenated string 26. When the matrix processing unit 56 concatenates multiple characters 25 as a concatenated string 26, the coordinate data of the concatenated string 26 is formed based on the coordinate data of the characters 25 that make up the concatenated string 26. Specifically, the minimum and maximum horizontal coordinates and the minimum and maximum vertical coordinates of the input image 62, among the coordinates of the four corners of the characters 25 that make up the concatenated string 26, become the coordinates of the four corners of the concatenated string 26. The matrix processing unit 56 calculates the orientation of the sides of the rectangle surrounding the concatenated string 26 from the coordinates of the four corners of the concatenated string 26. In Figure 37, a straight line 40N is shown along the orientation of the bottom edge of the rectangle surrounding the concatenated string 26 in the lower left. The matrix processing unit 56 calculates the angle θ between the straight line 40N (the orientation of the concatenated string 26, which is symbol 20) and the first direction 40H.

[0252] The matrix processing unit 56, which has calculated the angle θ, performs rotation correction on at least one of the coordinates of the input image 62 and the symbol 20 based on the angle θ. Specifically, the coordinates of the input image 62 and the symbol 20 (either one or both) are rotated so that the angle θ calculated from the corrected coordinates of the input image 62 and the symbol 20 becomes small (preferably zero). By performing rotation correction in this way, even if the first direction 40H of the AIM light 40 does not coincide with the row or column direction of the matrix attached to the object 11, the coordinates of the input image 62 and the symbol 20 with a small deviation of the first direction 40H relative to the row or column direction are obtained, thereby improving the reading accuracy of the symbol 20.

[0253] In addition to outputting information about tilted characters 25 in the input image 62, the machine learning model 61 may also be capable of concatenating multiple characters 25 that are in close proximity to each other to form a concatenated string 26. However, the machine learning model 61 alone cannot determine whether each of the multiple concatenated strings 26 is in the same row or column in a matrix. Therefore, the matrix processing unit 56 separately determines whether each of the multiple concatenated strings 26 is in the same row or is in the same column.

[0254] Next, referring to Figure 38, we will explain the first distance ΔV used to determine whether each of the multiple symbols 20 (here, concatenated strings 26) in the input image 62 is in the same row. For the sake of explanation, in Figure 38 and subsequent figures, we will assume that the first direction 40H of the Eima light 40 is parallel to the horizontal direction of the outer frame of the input image 62, and that the orientation of each concatenated string 26 is also parallel to the first direction 40H. The matrix processing unit 56 can handle an input image 62 that is essentially without tilt, as shown in Figure 38, by appropriately correcting the coordinates of the input image 62 and the symbols 20.

[0255] Figure 38 shows seven concatenated strings 26 as symbols 20. Hereafter, these concatenated strings 26 will be distinguished by the codes 26h, 26f, 26d, 26g, 26a, 26c, and 26n, respectively. The matrix processing unit 56 selects two of the multiple concatenated strings 26 in the input image 62 as the first symbol and two as the second symbol, and determines whether the first symbol and the second symbol are in the same row in the matrix. As an example, we will explain the case where the upper left concatenated string 26h is the first symbol and the upper right concatenated string 26f is the second symbol.

[0256] The matrix processing unit 56 determines the first reference point 43h for the first symbol, which is the concatenated string 26h. The first reference point 43h can be any single point in the coordinates contained in the concatenated string 26h, but here, as an example, the coordinates of the lower right corner of the concatenated string 26h are set as the first reference point 43h.

[0257] Figure 38 shows a first straight line 41 that passes through the first reference point 43h and is parallel to the first direction 40H of the Eima light 40. The matrix processing unit 56 calculates the first distance ΔV between this first straight line 41 and the second symbol, the concatenated string 26f. Specifically, a first reference point 43f (the lower right end in this case) is also defined for the second symbol, the concatenated string 26f, and the distance between the first straight line 41 and the first reference point 43f is calculated as the first distance ΔV.

[0258] The matrix processing unit 56 determines that the first symbol and the second symbol are on the same row if the first distance ΔV is sufficiently small (for example, below a threshold of 10 pixels). In Figure 38, the first distance ΔV between the concatenated string 26h (first symbol) and the concatenated string 26f (second symbol) is small, so it is determined that the concatenated string 26h (first symbol) is on the same row as the concatenated string 26f (second symbol).

[0259] The matrix processing unit 56 calculates and determines the first distance ΔV for all combinations of the concatenated strings 26 as the first symbol and the second symbol. FIG. 38 shows the first distance ΔV when the first symbol is the concatenated string 26h and the second symbols are the concatenated strings 26d and 26g. Since the first distance ΔV is large for the concatenated strings 26d and 26g (second symbols) with respect to the concatenated string 26h (first symbol), it is determined that the concatenated string 26h (first symbol) is located in a different row from the concatenated strings 26d and 26g (second symbols).

[0260] By calculating and determining the first distance ΔV for all combinations of two concatenated strings 26 each, in FIG. 38, the combinations of "concatenated string 26h and concatenated string 26f", "concatenated string 26d and concatenated string 26g", and "concatenated string 26a and concatenated string 26c" are respectively determined to be located in the same row, and all other combinations are determined to be located in different rows.

[0261] Next, referring to FIG. 39, the second distance ΔH used to determine whether each of the plurality of symbols 20 (concatenated strings 26) in the input image 62 is arranged in the same column will be described. As an example, the case where the concatenated string 26h at the upper left is the first symbol and the concatenated string 26f at the upper right is the second symbol will be described.

[0262] The matrix processing unit 56 determines the second reference point 44h for the concatenated string 26h which is the first symbol. The second reference point 44h may be any one of the coordinates included in the concatenated string 26h. Here, as an example, the coordinate of the upper left end of the concatenated string 26h is determined as the second reference point 44h.

[0263] FIG. 38 shows a second straight line 42 that passes through the second reference point 44h and intersects the first direction 40H. The matrix processing unit 56 calculates a second distance ΔH between this second straight line 42 and the concatenated character string 26f, which is the second symbol. Specifically, a second reference point 44f (the upper left end) is also defined for the concatenated character string 26f, which is the second symbol, and the distance between the second straight line 42 and the second reference point 44f is calculated as the second distance ΔH.

[0264] When the second distance ΔH is sufficiently small (for example, less than a threshold value such as 10 pixels), the matrix processing unit 56 determines that the first symbol and the second symbol are arranged in the same column. In FIG. 39, since the second distance ΔH between the concatenated character string 26h (the first symbol) and the concatenated character string 26f (the second symbol) is large, it is determined that the concatenated character string 26h (the first symbol) is located in a different column from the concatenated character string 26f (the second symbol).

[0265] The matrix processing unit 56 calculates and determines the second distance ΔH for all combinations of the concatenated character strings 26 as the first symbol and the second symbol. FIG. 39 shows the second distance ΔH when the first symbol is the concatenated character string 26h and the second symbols are the concatenated character strings 26d and 26g. As a result, since the second distance ΔH between the concatenated character string 26h (the first symbol) and the concatenated character string 26d (the second symbol) is small, it is determined that the concatenated character string 26h is arranged in the same column as the concatenated character string 26d (the second symbol). On the other hand, since the second distance ΔH between the concatenated character string 26h (the first symbol) and the concatenated character string 26g (the second symbol) is large, it is determined that the concatenated character string 26h is located in a different column from the concatenated character string 26g (the second symbol).

[0266] By calculating and determining the second distance ΔH for all combinations of two concatenated character strings 26, in FIG. 39, the combinations of "the concatenated character string 26h, the concatenated character string 26d, the concatenated character string 26a, the concatenated character string 26n" and "the concatenated character string 26f, the concatenated character string 26g, the concatenated character string 26c" are respectively determined to be located in the same column, and all other combinations are determined to be located in different columns.

[0267] The matrix processing unit 56 identifies the row and column in the matrix for each of the multiple symbols 20 (concatenated string 26) as described above. Based on the identified row and column, the output unit 12 outputs the symbol information represented by the multiple symbols 20 as matrix structured data (e.g., a CSV file).

[0268] In the above, the first reference point 43h and the second reference point 44h are assumed to be different, but the first reference point 43h and the second reference point 44h may be the same. Figure 40 shows the case where the first and second reference points are the same common reference point 45h (the coordinates of the center of the concatenated string 26h). As shown in Figure 40, even if the first and second reference points are the same common reference point 45h, we can consider the first straight line 41 and the second straight line 42 passing through the common reference point 45h. The matrix processing unit 56 can calculate the first distance ΔV and the second distance ΔH for each pair of concatenated strings 26 and identify the row and column in the matrix where each concatenated string 26 is located.

[0269] The positions of the first reference point 43h and the second reference point 44h in the concatenated string 26 should be determined according to the format information predetermined for the format of the concatenated string 26. For example, if it is predetermined that the concatenated string 26 will be written left-aligned, then the leftmost positions of the concatenated string 26 should be the first reference point 43h and the second reference point 44h.

[0270] When matrix processing is performed on symbols 20 attached to an object 11, it is preferable that format information relating to a matrix consisting of multiple symbols 20 is pre-registered by pre-setting. The format information relating to the matrix includes the number of rows and columns contained in the object 11. It is preferable that the format information further includes the format pattern of the object to be read by the matrix processing unit 56. Before performing work that includes matrix processing, the operator pre-configures the matrix processing. For example, pre-configuring the matrix processing is started by selecting the "List Settings" item from the setting menu screen displayed on the display unit 14 of the optical information reading device 10. When pre-configuring the matrix processing is started, a pre-configuration screen that accepts various pre-configurations relating to matrix processing is displayed on the display unit 14 (or the monitor of the host computer 19).

[0271] Figure 41 shows an example of a pre-configuration screen for matrix processing, specifically a list setting screen 46 and a column formatting screen 47. The display unit 14 first displays the list setting screen 46. The list setting screen 46 in Figure 41 includes a row number registration field 46a, a column number registration field 46b, and a column formatting button 46c.

[0272] The operator can specify the number of rows and columns of the matrix attached to the object 11 using the row number registration field 46a and the column number registration field 46b. For example, the operator can set the number of rows and columns by entering numerical values ​​in the row number registration field 46a and the column number registration field 46b, or by selecting numerical values ​​from a pull-down list. The operator can also set the number of rows or columns to "undefined". For example, if multiple symbols 20 arranged in the row direction of the matrix constitute one record with multiple data items, and multiple symbols 20 arranged in the column direction represent the data items of each record, the number of data items (number of columns) may be known, but the number of records (number of rows) may be unknown. In such cases, it is appropriate to set the unknown number to "undefined".

[0273] In Figure 41, the number of rows is set to "4" and the number of columns to "3". After setting the number of rows and columns, the user then operates the column format button 46c. When the column format button 46c is operated, the display on the display unit 14 transitions to the column format setting screen 47. The column format setting screen 47 displays a number of column format registration buttons corresponding to the number of columns set on the list setting screen 46. In Figure 41, three buttons are displayed: the 1st column format registration button 47a, the 2nd column format registration button 47b, and the 3rd column format registration button 47c. These buttons correspond to the columns (data items) in the row and column.

[0274] When any of the first column format registration buttons 47a, second column format registration buttons 47b, or third column format registration buttons 47c are pressed, the display unit 14 transitions to a screen where the format pattern of the target (symbol 20) to be read by the matrix processing unit 56 is registered for each column. On the screen for registering format patterns, the operator registers the format pattern for each column.

[0275] The screen for registering format patterns is similar to the format registration screen 38 in Figure 10, and includes a format registration field for registering the format of the string to be read, for example. However, if the object to be read is a symbol 20 other than a string, for example, an encoded code such as a barcode, it is preferable that the standard type of that encoded code can be registered.

[0276] Figure 42 shows an example of a registered format pattern 48 and an example of an input image 62 to be read, to which text and other elements have been added according to this format pattern 48.

[0277] Figure 42 shows format patterns 48 for three columns. In these format patterns 48, as in Figure 10, "A" represents an alphabet character and "9" represents a number. Corresponding to the first column ("Column 1"), a format for a string consisting of five digits is registered. Corresponding to the second column ("Column 2"), a format for a string consisting of three alphabet characters followed by three digits is registered. And for the third column ("Column 3"), instead of a string format, a coding code standard type called "CODEXXX" is registered. This indicates that "Column 3" has a coding code rather than a string, and that the appropriate code standard type for decoding that coding code is "CODEXXX". For example, multiple standard type options such as JAN code and CODE128 are available, and the operator should select the standard type corresponding to the coding code to be read from the options.

[0278] As described above, the format pattern 48 registered by the operator, along with information including the number of rows and columns, is acquired by the format information acquisition unit 59 in Figure 4 and stored as format information 65 in the storage unit 60 in Figure 5. Note that the format information 65 may also be acquired by estimation from the reading history information of the optical information reader 10, in addition to being registered by the operator.

[0279] If format information 65 containing a format pattern 48 as shown in Figure 42 is stored, the matrix processing unit 56 determines that the matrix contained in the object 11 shown in Figure 42 matches the format information 65. The matrix contained in the object 11 shown in Figure 42 has 4 rows and 3 columns, which matches the number of rows and columns set in the list setting screen 46 (Figure 41). The first column C1 is assigned a string consisting of 5 digits, which matches the setting for "Column 1". The second column C2 is assigned a string consisting of 3 alphabet characters followed by 3 digits, which matches the setting for "Column 2". The third column C3 is assigned an encoded code (in this case, a barcode). If the encoded code in the third column C3 can be decoded according to the "CODEXXX" standard, then the third column C3 will also match the setting for "Column 3". Figure 42 shows the decoded result of the barcode in the third column C3 (in this case, a code represented by a 3-digit number) in a callout.

[0280] The output unit 12 of the optical information reading device 10 outputs structured matrix data 49 that the matrix processing unit 56 has determined to match the format information 65, and stores it in the storage unit 60 (Figure 5). The user application 90 stored in the storage unit 60 can use the structured matrix data 49 as master data 92 for the matching process described later, by coordinating with the matrix processing unit 56.

[0281] The matching process is a process that checks whether the symbol information of the symbol 20 attached to the object 11 is included in the master data 92 that is pre-stored in the storage unit 60. The user application 90 may be used in the matching process in cooperation with the matrix processing unit 56. For example, in inventory confirmation work, which is a type of warehouse management work, the matching process may be performed to check whether the goods remaining in the warehouse match the information recorded on the inventory sheet that lists the inventory of the goods.

[0282] FIG. 43 shows an example in which the structured data 49 is used as the master data 92 in the collation process. Prior to performing the collation process, an operator uses the optical information reading device 10 to image an object 11 (e.g., an inventory ticket) to which a plurality of symbols 20 are attached as a matrix. The structured data 49 of the matrix attached to the object 11 is stored in the storage unit 60.

[0283] When the user application 90 (collation process application) is launched for the collation process, the user application 90 uses the stored structured data 49 as the master data 92 for the collation process. The user application 90 generates the collation data 93 with the checked flag item 93a added to this master data 92 (structured data 49) for the collation process. The checked flag item 93a is a new column-direction data item corresponding to each record (row-direction data) of the master data 92. The checked flag item 93a can switch between the state of "unchecked" (the confirmation flag is not set) and the state of "checked" (the confirmation flag is set) corresponding to each record.

[0284] In the collation process, the operator images the object 11 with the imaging unit 31 while the user application 90 as the collation process application is launched. In the collation process, the matrix processing unit 56 acquires the symbol information represented by the symbol 20 attached to the object 11 imaged by the imaging unit 31 and transmits it to the user application 90. The user application 90 checks whether the information acquired by the imaging unit 31 and the matrix processing unit 56 is included in the master data 92.

[0285] When the user application 90 confirms that the symbol information of the symbol 20 attached to the object 11 captured by the imaging unit 31 is included in the master data 92 (corresponding to one of the records), it sets the corresponding confirmed flag. In other words, in the matching data 93, the checked flag item 93a of the record corresponding to the symbol information confirmed to be included in the master data 92 switches from the "unchecked" state to the "checked" state (the confirmed flag is set).

[0286] The object 11 shown in Figure 43 has a barcode 23 that represents the numerical value "481" as symbol 20. The imaging unit 31 generates an input image 62 that includes the barcode 23. The matrix processing unit 56 detects the barcode 23 included in the input image 62 and obtains the symbol information represented by the barcode 23 (in this case, the code represented by the numerical value "481").

[0287] The user application 90 works in conjunction with the matrix processing unit 56 to check whether the information acquired by the imaging unit 31 and the matrix processing unit 56 is included in the master data 92. In this case, the code represented by the numerical value "481" is included in the master data 92, so the checked flag item 93a of the record (row-direction data) corresponding to that code in the matching data 93 is switched to the "checked" state.

[0288] For example, in inventory verification work, the worker performs a matching process using the user application 90. When the barcode 23 attached to the product remaining in the warehouse is captured by the imaging unit 31, if the checked flag item 93a of the matching data 93 is in the "checked" state, the worker can confirm that the product matches the product listed on the inventory sheet. If all the checked flag items 93a of the matching data 93 are in the "checked" state, the worker can confirm that the product remaining in the warehouse matches the information recorded on the inventory sheet listing the product inventory.

[0289] In the above example, the user application 90 sets the verified flag in the checked flag item 93a when the information acquired by the imaging unit 31 and the matrix processing unit 56 is included in the master data 92, but the matching of information may be confirmed by other methods. For example, the user application 90 may create a copy of the master data 92 as matching data 93, and delete the corresponding record from the matching data 93 when the information acquired by the imaging unit 31 and the matrix processing unit 56 is included in the master data 92. For example, in inventory confirmation work, when all records are deleted from the matching data 93, the worker can confirm that the goods remaining in the warehouse match the information written on the inventory sheet listing the inventory of the goods.

[0290] Next, referring to Figure 44, we will explain the process by which the matrix processing unit 56 removes information that does not match the format information 65 from the structured data 49. Figure 44 is a flowchart showing the flow of the process by which the matrix processing unit 56 removes information that does not match the format information 65.

[0291] As shown in Figure 44, in the pre-configuration step S10, which is performed prior to matrix processing, the "number of rows and columns setting" in step S01 and the "format setting" in step S02 are performed.

[0292] In step S01, "Setting the number of rows and columns," the operator sets the number of rows and columns included in the object 11, as shown in Figure 41. In step S02, "Setting the format," the operator sets a format pattern 48 for each column, as shown in Figure 42. The format information acquisition unit 59 (Figure 4) of the optical information reading device 10 acquires format information 65, including the number of rows and columns set in advance, and stores it in the storage unit 60 (Figure 5).

[0293] When matrix processing is performed after the above-described pre-configuration has been carried out, the matrix processing unit 56 groups the symbols 20 that are in the same row in the input image 62 in step S75 of Figure 44. In determining whether or not multiple symbols 20 are in the same row, a determination based on a first distance ΔV, such as that shown in Figure 38, may be performed.

[0294] The number of symbols 20 in the same row corresponds to the number of columns in that row (group). In step S76 of Figure 44, the matrix processing unit 56 compares the number of columns in each group with the number of columns set in step S01. If there is a group whose number of columns differs from the set number, that group (row) is deleted.

[0295] Next, in step S77, the matrix processing unit 56 groups the symbols 20 that are in the same column in the input image 62. In determining whether multiple symbols 20 are in the same column, a determination based on a second distance ΔH, such as shown in Figure 39, may be performed.

[0296] The number of symbols 20 in the same column corresponds to the number of rows in that column (group). In step S78 of Figure 44, the matrix processing unit 56 compares the number of rows in each group with the number of rows set in step S01. If there is a group whose number of rows differs from the set number, that group (column) is deleted.

[0297] As described above, the matrix processing unit 56 determines whether the rows and columns of the matrix included in the object 11 match the format information 65, and deletes the rows and columns that do not match the format information 65. The output unit 12 of the optical information reading device 10 outputs structured data 49 of the matrix that the matrix processing unit 56 has determined to match the format information 65. At this time, the rows and columns deleted by the matrix processing unit 56 are not included in the structured data 49. Therefore, based on the determination result of the matrix processing unit 56, the output unit 12 outputs the structured data 49 excluding the rows and columns that do not match the format information 65.

[0298] Referring to Figure 45, an example of when information that does not match the format information 65 is excluded from the structured data 49 will be explained. The object 11 shown in Figure 45 has a matrix with 5 columns and an unspecified number of rows. Each column in the matrix of object 11 contains numbers, letters, and kanji characters.

[0299] In Figure 45, it is assumed that the number of rows is set to "undefined" and the number of columns to "5" in the pre-configuration for the matrix. Furthermore, the format pattern 48 for each column is set to be a string consisting of numbers or letters.

[0300] The matrix processing unit 56 obtains information for each row by reading the symbol information of the string that matches the format pattern 48. For example, the information for one row in CSV format data with commas as delimiters is represented in the format "1,AAA,12,200,2400". This row has 5 columns and matches the format information 65.

[0301] The object 11 in Figure 45 has a column (header) that represents the name of the data item, and one of the data item names contains the string "No.". The matrix processing unit 56 determines that the string "No." matches the format pattern 48 and obtains the information of the row represented as "No.," in CSV format. However, this row has only one column, so it does not match the format information 65. Therefore, the matrix processing unit 56 deletes the row represented as "No.,". Consequently, the structured data 49 output by the output unit 12 will have rows that do not match the format information 65 ("No.,") removed. As a result, the output unit 12 can output structured data 49 that matches the format information 65 defined in the pre-configuration.

[0302] In the matrix processing described above, the matrix processing unit 56 determined whether multiple symbols 20 are in the same column based on the second distance ΔH (Figure 39) in the input image 62. However, whether multiple symbols 20 are in the same column may also be determined using the format pattern 48.

[0303] For example, if the object 11 is imaged from a distance, multiple symbols 20 that are arranged in the same column in the object 11 may appear not to be arranged in the same column in the input image 62 due to perspective. In such cases, the column direction is correctly determined by using the format pattern 48 to determine whether the multiple symbols 20 are arranged in the same column.

[0304] Figure 46 illustrates matrix processing that takes perspective into account. Object 11 is associated with a matrix of multiple symbols 20 (nine in this case) arranged in both row and column directions. However, when object 11 is imaged from a distance, the symbols 20 on the near side (lower side of Figure 46) in the input image 62 appear to be spread out horizontally due to perspective, compared to the symbols 20 on the far side (upper side of Figure 46), and may not appear to be arranged in the same column as the symbols 20 on the far side.

[0305] The object 11 shown in Figure 46 is actually a rectangular shape, but because it is imaged from a distance, it appears trapezoidal in the input image 62. For illustrative purposes, the shape of the object 11 is shown here as being significantly distorted from its original shape in the input image 62.

[0306] In such cases, the matrix processing unit 56 can use the format pattern 48 to determine whether multiple symbols 20 are in the same column. Specifically, first, similar to Figure 38, the matrix processing unit 56 determines the row direction by calculating the first distance ΔV (identifying multiple symbols 20 in the same row). The effect of perspective on the first distance ΔV is small, and the matrix processing unit 56 can correctly determine the row direction (the direction of the first line 41).

[0307] After identifying one of the row and column directions (the row direction in the example of Figure 46), the matrix processing unit 56 groups multiple symbols that match the format pattern 48 in the input image 62 as a symbol group 20G. The straight line connecting these symbol groups 20G is defined as the second straight line 42. The matrix processing unit 56 then identifies the other of the row and column directions by calculating the second distance ΔH between the second straight line 42 and each string (identifying multiple symbols 20 that are in the same column in the example of Figure 46). Each symbol 20 is at least one of a string or a code (such as a barcode or two-dimensional code) in which symbol information is encoded.

[0308] The line connecting the symbol group 20G refers to the line connecting multiple symbols (such as strings of characters) included in the symbol group 20G. If the symbol group 20G contains two symbols, the line connecting the reference points (e.g., the center) of each symbol becomes the second line 42. If the symbol group 20G contains three or more symbols, the most appropriate second line 42 should be drawn using the least squares method or similar, so that the distance between the second line 42 and the reference point of each symbol is as small as possible.

[0309] Multiple symbols (such as strings of characters) that match the pre-configured format pattern 48 are likely to be arranged in the same column. Therefore, by identifying multiple symbols 20 arranged in the same column in this manner, even when the object 11 is imaged from a distance, the matrix processing unit 56 can correctly acquire the symbol information represented by the multiple symbols 20, which are attached as a matrix aligned in the row and column directions, as structured data 49.

[0310] For example, if symbol 20 is a string, and the format of that string is specified as format pattern 48, the matrix processing unit 56 can determine that multiple strings matching the format specified by format pattern 48 are strings that should be in the same column. The matrix processing unit 56 may also use format pattern 48 to determine whether multiple symbols 20 are in the same column even if symbol 20 is not a string. For example, multiple symbols 20 representing encoded codes (e.g., barcodes) that match the standard type (e.g., CODE128) specified by format pattern 48 should be determined to be in the same column. In this case, the calculation of the second distance ΔH does not need to be performed. In the above example, the matrix processing unit 56 was described as determining the row direction by the direction indicated by the EIM light 40 and then determining the column direction using format pattern 48, but this is not the only way. The matrix processing unit 56 may also determine the column direction by the direction indicated by the EIM light 40 and then determine the row direction using format pattern 48. In this case, the matrix processing unit 56 can determine the row direction by sequentially searching for symbols 20 that match the format, number of digits, or code type defined for each column.

[0311] Furthermore, if the object 11 appears trapezoidal in the input image 62, as shown in Figure 46, the matrix processing unit 56 may perform a projection transformation on the input image 62. A projection transformation is a process that converts the original coordinate system to another coordinate system. In this case, the projection transformation converts the coordinates of each symbol within the trapezoidal object 11 to the coordinates within the original rectangular object 11.

[0312] If object 11 appears as a trapezoid in the input image 62, then the virtual line 40L, which is an extension of the left side 11L of object 11 in the input image 62, and the virtual line 40R, which is an extension of the right side 11R, are considered to intersect at vanishing point 99 at infinity. Similarly, multiple second lines 42 connecting the symbols 20 arranged in the column direction are also considered to intersect at vanishing point 99.

[0313] The matrix processing unit 56 then obtains the coordinates of the four corners of the object 11 in the input image 62. For example, the matrix processing unit 56 can identify the coordinates of the four corners of the object 11 by detecting positions (edges) in the input image 62 where the contrast change is large. If the matrix attached to the object 11 is a matrix enclosed by grid lines, the coordinates of the four corners of the grid lines may be used instead of the coordinates of the four corners of the object 11.

[0314] The matrix processing unit 56 calculates the projection transformation matrix necessary for the projection transformation so that the coordinates of the four corners of the object 11 are transformed into the coordinates of the four corners of the input image 62. In the coordinate system after projection transformation using the calculated projection transformation matrix, the virtual line 40L, the virtual line 40R, and the multiple second lines 42 become parallel, and the column direction can be determined using the second distance ΔH in the same way as in Figure 39.

[0315] Alternatively, the matrix processing unit 56 may consider the angle between symbols 20 instead of the distance between them to determine the row direction (identify symbols 20 in the same row) and the column direction (identify symbols 20 in the same column).

[0316] Figure 47 illustrates matrix processing that takes into account the angles between symbols 20. The matrix processing unit 56 draws a first line 41 parallel to the first direction 40H of the AIM light 40 and a second line 42 parallel to the second direction 40V that intersects the first direction 40H, so that they pass through the reference point (e.g., the center) of the symbols 20.

[0317] The matrix processing unit 56 then calculates the first angle θx formed by the line segment connecting the reference points of the two symbols 20 and the first straight line 41. If this first angle θx is sufficiently small (for example, below a predetermined threshold), it is determined that the two symbols 20 are in the same row.

[0318] The matrix processing unit 56 also calculates a second angle θy formed by the line segment connecting the reference points of the two symbols 20 and the second line 42. If this second angle θy is sufficiently small (for example, below a predetermined threshold), it is determined that the two symbols 20 are in the same column. In addition, both the distance and angle between the symbols 20 may be considered when determining the row direction and the column direction.

[0319] Incidentally, the embodiments are illustrative and not restrictive in all respects. The scope of this disclosure is indicated by the claims rather than the foregoing description, and all modifications within the meaning and scope of equivalence to the claims are intended. Of the configurations described in the embodiments, those described as an aspect of this disclosure in “Means for Solving the Problem” are optional and may be deleted or modified as appropriate. [Industrial applicability]

[0320] This disclosure provides an optical information reading device, which has industrial applicability. [Explanation of Symbols]

[0321] 10 Optical Information Reading Device 11. Object 12 Output section 13 Imaging Module 14 Display section 15 Control section 18 Trigger Keys 19 Host computer 20 Symbols 31 Imaging Unit 32. Eima light irradiation section 40 Eima Light 48 format patterns 49 Structured Data 53 Information Processing Department 54 String Processing 55 Relative reading processing unit 56 Matrix Processing Unit 57 Designated Reception Department 58 Reading target setting section 60 Storage section 61 Machine Learning Models 62 Input Images 63 Setting Images 64 operational images 65 Format Information 74. Screen for setting the target to read 76. Reading target setting information 90 User Applications 92 Master Data 100 Information Processing Systems

Claims

1. An optical information reading device that images an object to which multiple symbols are attached as a matrix in which multiple symbols are aligned in the row direction and in the column direction intersecting the row direction, and reads the symbol information represented by the multiple symbols, An EIM light irradiation unit that irradiates EIM light extending in a first direction intersecting the irradiation direction toward the object, An imaging unit having an imaging field of view oriented toward the aforementioned irradiation direction, imaging a region within the imaging field of view and generating an input image that includes the symbol attached to the object, A matrix processing unit that performs matrix processing based on the aforementioned input image, Based on the results of the matrix processing by the matrix processing unit, an output unit outputs the symbol information represented by the plurality of symbols attached to the object as structured data of the structured matrix, Equipped with, The matrix processing unit described above is By performing image processing on the input image, the plurality of symbols contained in the input image are detected. Two of the aforementioned symbols are designated as the first symbol and the second symbol. The first distance between the first straight line, which passes through a first reference point defined for the first symbol and is parallel to the first direction, and the second symbol is calculated. The second distance between the second straight line, which passes through the second reference point defined for the first symbol and intersects the first direction, and the second symbol is calculated. Based on the first and second distances between the first and second symbols, it is determined whether the first and second symbols are in the same row in the matrix and whether they are in the same column in the matrix, thereby identifying the row and column in the matrix where each of the plurality of symbols is located. The output unit outputs the symbol information represented by the plurality of symbols as structured data of the matrix, based on the rows and columns identified by the matrix processing unit, in an optical information reading device.

2. An optical information reading device that images an object to which multiple symbols are attached as a matrix in which multiple symbols are aligned in the row direction and in the column direction intersecting the row direction, and reads the symbol information represented by the multiple symbols, An imaging unit that captures an area within the imaging field of view and generates an input image that includes the symbol attached to the object, A display unit that displays the aforementioned input image and superimposes a virtual aimer extending in a first direction for determining the aiming position onto the input image, A matrix processing unit that performs matrix processing based on the aforementioned input image, Based on the results of the matrix processing by the matrix processing unit, an output unit outputs the symbol information represented by the plurality of symbols attached to the object as structured data of the structured matrix, Equipped with, The matrix processing unit described above is By performing image processing on the input image, the plurality of symbols contained in the input image are detected. Two of the aforementioned symbols are designated as the first symbol and the second symbol. The first distance between the first straight line, which passes through a first reference point defined for the first symbol and is parallel to the first direction, and the second symbol is calculated. The second distance between the second straight line, which passes through the second reference point defined for the first symbol and intersects the first direction, and the second symbol is calculated. Based on the first and second distances between the first and second symbols, it is determined whether the first and second symbols are in the same row in the matrix and whether they are in the same column in the matrix, thereby identifying the row and column in the matrix where each of the plurality of symbols is located. The output unit outputs the symbol information represented by the plurality of symbols as structured data of the matrix, based on the rows and columns identified by the matrix processing unit, in an optical information reading device.

3. The system further comprises a format information acquisition unit that acquires format information including the number of rows and the number of columns included in the object, The matrix processing unit determines whether the identified row and column match the format information. The optical information reading device according to claim 1 or 2, wherein the output unit outputs the structured data, excluding the rows and columns that do not match the format information, based on the determination result of the matrix processing unit.

4. The optical information reading device according to claim 3, wherein the format information includes a format pattern to be read for each of the columns.

5. The optical information reading device according to claim 4, wherein the matrix processing unit determines one of the row direction and the column direction by calculating the first distance, and then determines the other of the row direction and the column direction by calculating the second distance using a straight line connecting a group of symbols that match the format pattern as the second straight line.

6. The system further comprises a storage unit that stores a user application used for matching processing, which cooperates with the imaging unit and the matrix processing unit, and the structured data. The optical information reading device according to claim 1 or 2, wherein the user application uses the structured data as master data for the matching process and checks whether the information acquired by the imaging unit and the matrix processing unit is included in the master data.

7. The aforementioned multiple symbols are strings, The matrix processing unit described above is An optical information reading device according to claim 1 or 2, which detects multiple characters and obtains multiple concatenated strings by concatenating the multiple characters based on their positions and sizes.

8. The matrix processing unit sorts the detected plurality of characters in the order they are arranged in the X direction corresponding to the first direction and in the Y direction perpendicular to the X direction, and selects at least two characters that are arranged consecutively along the X direction or the Y direction as subjects for determining whether or not they should be concatenated, according to claim 7.

9. The matrix processing unit described above is The orientation of at least one of the aforementioned multiple symbols is calculated, The angle between the calculated direction and the first direction is calculated, The optical information reading device according to claim 1 or 2, wherein rotation correction is performed on at least one of the coordinates of the input image and the symbol based on the calculated angle.

10. An optical information reading device that images an object to which multiple symbols are attached as a matrix in which multiple symbols are aligned in the row direction and in the column direction intersecting the row direction, and reads the symbol information represented by the multiple symbols, An EIM light irradiation unit that irradiates EIM light extending in a first direction intersecting the irradiation direction toward the object, An imaging unit having an imaging field of view oriented toward the aforementioned irradiation direction, imaging a region within the imaging field of view and generating an input image that includes the symbol attached to the object, A matrix processing unit that performs matrix processing based on the aforementioned input image, Based on the results of the matrix processing by the matrix processing unit, an output unit outputs the symbol information represented by the plurality of symbols attached to the object as structured data of the structured matrix, A storage unit that stores format information including the format pattern to be read for each column of the aforementioned matrix, Equipped with, The matrix processing unit described above is By performing the matrix processing on the input image, the plurality of symbols contained in the input image are detected. Two of the aforementioned symbols are designated as the first symbol and the second symbol. The first distance between the first straight line, which passes through a first reference point defined for the first symbol and is parallel to the first direction, and the second symbol is calculated. Based on the first distance between the first symbol and the second symbol, it is determined whether the first symbol and the second symbol are located in the same row and the same column in the matrix, thereby identifying the one of the same row and the same column in the matrix for each of the plurality of symbols. The symbol that matches the format pattern is estimated to be located in the other of the same row and the same column in the matrix, The output unit outputs the symbol information represented by the plurality of symbols as structured data of the matrix, based on the rows and columns identified by the matrix processing unit, in an optical information reading device.

11. An optical information reading device that images an object to which multiple symbols are attached as a matrix in which multiple symbols are aligned in the row direction and in the column direction intersecting the row direction, and reads the symbol information represented by the multiple symbols, An imaging unit that captures an area within the imaging field of view and generates an input image that includes the symbol attached to the object, A display unit that displays the aforementioned input image and superimposes a virtual aimer extending in a first direction for determining the aiming position onto the input image, A matrix processing unit that performs matrix processing based on the aforementioned input image, Based on the results of the matrix processing by the matrix processing unit, an output unit outputs the symbol information represented by the plurality of symbols attached to the object as structured data of the structured matrix, A storage unit that stores format information including the format pattern to be read for each column of the aforementioned matrix, Equipped with, The matrix processing unit described above is By performing the matrix processing on the input image, the plurality of symbols contained in the input image are detected. Two of the aforementioned symbols are designated as the first symbol and the second symbol. The first distance between the first straight line, which passes through a first reference point defined for the first symbol and is parallel to the first direction, and the second symbol is calculated. Based on the first distance between the first symbol and the second symbol, it is determined whether the first symbol and the second symbol are located in the same row and the same column in the matrix, thereby identifying the one of the same row and the same column in the matrix for each of the plurality of symbols. The symbol that matches the format pattern is estimated to be located in the other of the same row and the same column in the matrix, The output unit outputs the symbol information represented by the plurality of symbols as structured data of the matrix, based on the rows and columns identified by the matrix processing unit, in an optical information reading device.

Images