Optical information reader

Abstract

To restore an image including a code of wide pixel resolution while improving a processing speed, as a neural network structure dedicated to restoration of images including codes.SOLUTION: A processor inputs a read image enlarged or reduced so that pixel resolution of a module constituting a code falls within a predetermined range, to a neural network configured with a structure and parameters stored in a storage unit, to try to restore the read image, and then executes decoding processing on the restored read image.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention relates to an optical information reading device that optically reads information. [Background technology]

[0002] In recent years, importance has been placed on traceability, which allows tracking of the distribution route of goods from the manufacturing stage to the consumption stage or disposal stage, and code readers for this purpose have become widespread.In addition to traceability, code readers are also used in various other fields.

[0003] Generally, a code reader is configured to use a camera to capture a barcode, two-dimensional code, or other code attached to a workpiece, extract the code contained in the resulting image using image processing, digitize it, and decode it to read the information.Since it is a device that reads information optically, it is also called an optical information reading device.

[0004] Known examples of this type of optical information reading device include one equipped with a machine learning device that learns a model structure that shows the relationship between a code image acquired by a robot's visual sensor and an ideal code image, as disclosed in Patent Document 1. Patent Document 1 describes that the results of learning by the machine learning device are applied to a code image acquired by a visual sensor during operation, thereby restoring the image to one suitable for reading. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6683666 Summary of the Invention [Problem to be solved by the invention]

[0006] In order to fully capture the features of the code and restore it to an image suitable for reading, it is necessary to extract features from a range that covers a certain number of modules. For example, if a feature value obtained from a neural network is a value extracted from a range of 6x6 modules, and an image was taken with one module consisting of 20 pixels, the neural network must be designed to calculate a feature value aggregated from a range of 120x120 pixels.

[0007] In other words, the wider the pixel range you want to cover, the deeper the neural network hierarchy must be. For example, if you want to aggregate feature values ​​from the 120x120 pixel range mentioned above, you will need six convolutional layers.

[0008] However, since the code is composed of a random arrangement of white and black modules, the relationship between pixel values ​​over a wide range is weak, and expanding the pixel range beyond a certain point hardly improves the restoration effect.

[0009] The present invention has been made in view of the above points, and its purpose is to provide a neural network structure specialized for the restoration of images containing codes, which improves processing speed while enabling the restoration of images containing codes with a wide range of pixel resolution. [Means for solving the problem]

[0010] To achieve the above object, one aspect of the present disclosure can be based on an optical information reading device that reads a code attached to a workpiece. The optical information reading device includes a camera that photographs the code and generates a read image, a memory unit that stores the structure and parameters of a neural network that has been pre-generated by machine learning a plurality of defective images having portions unsuitable for reading the code and a plurality of ideal images corresponding to the plurality of defective images, where the defective images and the ideal images each have a pixel resolution of modules that make up the code within a specific range, and a processor that inputs the read image generated by the camera, enlarged or reduced so that the pixel resolution of modules that make up the code falls within the specific range, into the neural network configured with the structure and parameters stored in the memory unit, attempts to repair the read image in accordance with the structure and parameters stored in the memory, and performs a decoding process on the repaired read image.

[0011] According to this configuration, when generating the neural network structure and parameters, defective images and ideal images in which the pixel resolution of the modules that make up the code falls within a specific range are machine-learned. The specific range includes pixel resolutions that provide a high repair effect and excludes pixel resolutions that provide little improvement in repair effect, and can be set on the premise of repairing images that include codes made up of multiple modules. This allows for improved processing speed as a neural network structure specialized for repairing images that include codes.

[0012] Furthermore, if the pixel resolution of the modules that make up the code in the scanned image generated by the camera is outside the specified range, the scanned image is enlarged or reduced so that it falls within the specified range before being input into the neural network, thereby making it possible to repair images that include codes with a wide range of pixel resolutions.

[0013] Note that the arrangement of fixed module patterns, such as finder patterns contained in the code, and the rectangular shape of the code as a whole can be broad-range features, but if you only roughly separate the modules from the background, you can sufficiently repair the code using only narrow-range features without using broad-range features.

[0014] In another aspect of the present disclosure, the structure and parameters of the neural network are generated in advance by machine learning the defective image and the ideal image, in which the number of pixels of the modules that make up the code is within a specific range, and the processor can input to the neural network the scanned image generated by the camera, which has been enlarged or reduced so that the number of pixels of the modules that make up the code is within the specific range.

[0015] This configuration allows the neural network to be generated with a specific pixel count range, making it more specialized for repairing images containing code. The specific pixel count range can be, for example, between 4 and 6 PPC, but it can also be less than 4 PPC or more than 6 PPC. PPC is a unit that indicates how many pixels (picture elements) one module is.

[0016] Furthermore, by limiting the specific range to 4 PPC or more and 6 PPC or less for machine learning, for example, it is possible to eliminate unnecessary scale variations from the images used for learning, thereby optimizing the neural network structure.

[0017] In another aspect of the present disclosure, the image processing device further includes a filter processing unit that applies an image processing filter to the scanned image generated by the camera before enlarging or reducing the scanned image, thereby enabling appropriate image processing to be performed before enlarging or reducing the scanned image, resulting in accurate restoration of the scanned image. Examples of the image processing filter include a noise reduction filter.

[0018] In another aspect of the present disclosure, the optical information reading device further includes a tuning execution unit that, when setting up the optical information reading device, generates a plurality of read images with different magnification ratios or a plurality of read images with different reduction ratios, inputs the generated read images to the neural network to attempt to repair each read image, performs a decoding process on each repaired read image to determine a reading margin, and specifies the magnification ratio or reduction ratio of a read image for which the determined reading margin is higher than a predetermined value as the magnification ratio or reduction ratio to be used during operation. The processor can enlarge or reduce the read image to the magnification ratio or reduction ratio specified by the tuning execution unit when operating the optical information reading device.

[0019] In other words, if the specific range has a certain width, it is conceivable that the reading margin will change if the magnification or reduction ratio is changed even within that specific range. With this configuration, the magnification or reduction ratio of the read image during operation can be specified so that the reading margin is higher than the predetermined value, thereby improving processing speed and reading accuracy.

[0020] In another aspect of the present disclosure, the tuning execution unit identifies the magnification or reduction ratio of the read image with the highest reading margin among the reading margins of each read image as the magnification or reduction ratio to be used during operation, thereby further improving processing speed and reading accuracy.

[0021] In another aspect of the present disclosure, the storage unit stores the structures and parameters of multiple neural networks, each having a different number of layers or a different number of image processing filters, in association with a code condition. The optical information reading device further includes a tuning execution unit that sets a code condition to be included in a read image generated by the camera, reads from the storage unit the structure and parameters of a neural network associated with the set code condition, and identifies the neural network to be used during operation. The processor can attempt to repair the read image using the neural network identified by the tuning execution unit during operation of the optical information reading device.

[0022] That is, the optimal number of layers or filters of the neural network can be determined in advance based on the code conditions, particularly PPC, and can be associated with them. If the code conditions are specified when setting up the optical information reader, the structure and parameters of the neural network associated with the code conditions can also be specified. Therefore, repair can be attempted using the neural network structure and parameters that are optimal for the code conditions, thereby improving reading accuracy.

[0023] In another aspect of the present disclosure, the tuning execution unit sets code conditions to be included in the scanned image generated by the camera, acquires the number of pixels of the module that meets the set code conditions, and if the acquired number of pixels is within the specific range, prevents the processor from enlarging or reducing the module, so that the module can be enlarged or reduced only when the number of pixels of the module is outside the specific range.

[0024] In another aspect of the present disclosure, the storage unit is configured to store multiple parameter sets with different enlargement ratios or multiple parameter sets with different reduction ratios, and the processor can apply any one of the multiple parameter sets stored in the storage unit to execute the decoding process. This allows, for example, when the number of pixels in a code module changes, the optical information reader can be operated by applying a parameter for the enlargement ratio or reduction ratio corresponding to the number of pixels.

[0025] In another aspect of the present disclosure, the processor searches for a code in the enlarged or reduced scanned image based on information for identifying the code, extracts an area containing the searched code as a code candidate area where a code is likely to exist, inputs a partial image corresponding to the extracted code candidate area into a neural network configured with the structure and parameters stored in the memory unit, attempts to repair the partial image in accordance with the structure and parameters stored in the memory unit, and performs a decoding process on the repaired partial image.

[0026] This configuration allows the size of the image input to the neural network to be reduced. This reduces the computational load on the processor, thereby enabling faster processing. Furthermore, because the partial image corresponds to an area where a code is likely to exist, it is possible to create an image that includes all the information necessary for reading, i.e., the entire code. Because this image including the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

[0027] In another aspect of the present disclosure, the image restoration system further includes an image restoration unit that inputs a scanned image to a neural network and performs an inference process to generate a restored image by restoring the scanned image; a first decoding processing unit that performs a first decoding process on the scanned image generated by the camera; and a second decoding processing unit that performs a second decoding process on the restored image generated by the image restoration unit. The second decoding processing unit extracts a code candidate area from the scanned image and sends a trigger signal to the image restoration unit to execute the inference process. Upon receiving the trigger signal from the second decoding processing unit, the image restoration unit inputs a partial image corresponding to the code candidate area to the neural network and executes an inference process to generate a restored image by restoring the partial image. The second decoding processing unit can determine grid positions indicating the positions of each cell of the code based on the restored image and decode the restored image based on the determined grid positions. In other words, the first decoding processing unit decodes a normal read image that has not been subjected to inference processing, the image restoration unit generates a restored image through inference processing, and the second decoding processing unit decodes the restored image restored through inference processing, thereby constructing processing circuits suited to each and increasing the processing speed.In addition, the image restoration unit only needs to restore a partial image corresponding to the code candidate area, which enables further increase in processing speed.

[0028] In another aspect of the present disclosure, the inference processing by the image restoration unit and the first decoding processing by the first decoding processing unit are performed in parallel, thereby increasing the processing speed. Furthermore, at least a portion of the period during which the second decoding processing unit is performing the second decoding processing includes a pause period during which the inference processing by the image restoration unit is not performed, thereby reducing the load on the image restoration unit and suppressing heat generation. In this case, the second decoding processing unit may be configured with multiple cores.

[0029] In another aspect of the present disclosure, the first decoding processing unit can execute the first decoding process in a shorter time than the time required for the second decoding process. Furthermore, the setting unit can switch whether or not to execute the second decoding process, i.e., whether or not to execute decoding of the restored image. The second decoding process, which executes decoding of the restored image, takes longer than the first decoding process, which executes decoding of a normal scanned image. Given this premise, when the second decoding process is set to be executed, a decoding timeout period (decoding time limit) longer than the time required for executing the second decoding process can be set. On the other hand, when the second decoding process is set not to be executed, a decoding timeout period shorter than the time required for executing the second decoding process can be set. Therefore, if the decoding of the restored image exceeds the decoding timeout period, the process can quickly transition to decoding of the next captured image.

[0030] A decoding processing unit according to another aspect of the present disclosure can perform a second decoding process on a restored image in parallel with a first decoding process on a scanned image generated by a camera, thereby increasing the processing speed even when scanned images and restored images are mixed. When the setting unit sets the decoding processing unit not to perform the second decoding process, the decoding processing unit can allocate processing resources used for the second decoding process to the first decoding process. Processing resources include, for example, memory usage areas and cores constituting a multi-core CPU. When a memory area or core is previously set as a processing resource used for the second decoding process, setting the second decoding process not to be performed allows the memory area or core to be used for the first decoding process, thereby further increasing the speed of the first decoding process.

[0031] A decoding processing unit according to another aspect of the present disclosure extracts a code candidate area from a scanned image that is likely to contain a code. An image restoration unit inputs a partial image corresponding to the code candidate area into a neural network and generates a restored image by restoring the partial image. A decoding processing unit determines grid positions based on the restored image generated by the image restoration unit and decodes the restored image based on the determined grid positions. Since generating a restored image often takes time, the time required to generate the restored image can be reduced by extracting a code candidate area based on, for example, feature quantities that indicate code resemblance before generating the restored image. Furthermore, since the grid positions can be determined with high accuracy based on the restored image, reading performance is improved.

[0032] In another aspect of the present disclosure, a tuning process is performed to determine optimal imaging conditions and decoding conditions and to set the size of the code to be read. The decoding processing unit can determine the size of an extracted image containing a code candidate area from the scanned image based on the code size set in the tuning process. By enlarging or reducing the extracted image to a predetermined size and inputting it into a neural network, the PPC of the code image input to the neural network can be kept within a predetermined range, thereby stably achieving the image restoration effect of the neural network. Furthermore, although the code size to be read may vary, the size of the code ultimately input to the neural network can be fixed, thereby maintaining a constant balance between high processing speed and ease of decoding.

[0033] A camera according to another aspect of the present disclosure receives light that passes through a first polarizing plate and is reflected from the code via a second polarizing plate, generating a read image with lower contrast than when the light is not passed through the first and second polarizing plates. By using a polarizing plate, the specular reflection component of the workpiece is removed, resulting in a read image with reduced influence of the specular reflection component. While polarizing plates are suitable for images with a high specular reflection component, such as when capturing an image of a metal workpiece, reduced light intensity can cause the read image to become dark and the contrast to decrease. In such cases, reading performance can be improved by converting the image into a high-contrast restored image using neural network inference processing.

[0034] In another aspect of the present disclosure, the tuning execution unit can set a plurality of imaging conditions and code conditions including first imaging conditions and code conditions and second imaging conditions and code conditions. The decoding processing unit can input a read image generated under the first imaging conditions and code conditions to a neural network to generate a repaired image, and if decoding of the repaired image fails, input a read image generated under second imaging conditions and code conditions different from the first imaging conditions and code conditions to the neural network to generate a repaired image, and decode the repaired image. [Effects of the Invention]

[0035] As described above, a neural network generated in advance by machine learning defective images and ideal images in which the pixel resolution of the modules making up the code falls within a specific range can be input with a scanned image that has been enlarged or reduced so that the pixel resolution of the modules making up the code falls within a specific range, and the neural network can then attempt to repair the scanned image. This improves processing speed as a neural network structure specialized for repairing images that include codes, while also making it possible to attempt to repair images that include codes with a wide range of pixel resolution. [Brief explanation of the drawings]

[0036] [Figure 1]FIG. 10 is a diagram illustrating the stationary optical information reader during operation. [Figure 2] FIG. 1 is a perspective view of a stationary optical information reading device. [Figure 3] FIG. 1 is a block diagram of an optical information reader. [Figure 4] FIG. 1 is a perspective view of a handheld optical information reading device. [Figure 5] FIG. 2 is a diagram illustrating each unit configured by a processor. [Figure 6] FIG. 1 is a conceptual diagram of a neural network. [Figure 7] 1 is a flowchart showing an example of a basic procedure for learning a neural network. [Figure 8] A pair of defective and ideal images is shown, where (A) is an example of the defective image and (B) is an example of the ideal image. [Figure 9] FIG. 1 is a conceptual diagram of a convolutional neural network used for image conversion. [Figure 10] 10 is a flowchart showing an example of the procedure of a tuning process performed when setting up an optical information reader. [Figure 11] 10 is a flowchart illustrating an example of a decoding process procedure before a reduction ratio and an enlargement ratio are determined. [Figure 12] 10 is a flowchart illustrating an example of a decoding process procedure after a reduction ratio and an enlargement ratio are determined. [Figure 13] 1A shows an example of a scanned image generated by a camera, and FIG. 1B shows an example of an image after image processing filtering. [Figure 14] 10A shows an example of an image after reduction processing, and FIG. 10B shows an example of an image after code region extraction. [Figure 15] FIG. 10 is a diagram showing an example of restoration of a scanned image using a neural network. [Figure 16] FIG. 10 is a diagram illustrating an example of a decoding process procedure according to another embodiment. [Figure 17] 4 is a time chart of the decoding process by the first decoding processing unit and the second decoding processing unit. [Figure 18]10 is a time chart of a decoding process by a decoding processing unit. [Figure 19] 10 is a flowchart showing an example of a first decoding process and a second decoding process. [Figure 20] 10A to 10C are diagrams illustrating example images of each process. [Figure 21] 10A and 10B are diagrams illustrating a case where a plurality of neural networks are used to deal with black and white inversion of a code. [Figure 22] FIG. 10 is a diagram illustrating a case where one neural network is used to handle black and white inversion of a code. [Figure 23] 10 is a graph showing the relationship between contrast and matching level. [Figure 24] 10 is a flowchart illustrating an example of a process for calculating a matching level. [Figure 25] FIG. 4 is a diagram illustrating an example of a user interface displayed on a display unit. [Figure 26] FIG. 10 is a diagram showing an example in which an imaging element with an AI chip is used. DETAILED DESCRIPTION OF THE INVENTION

[0037] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present invention, its applications, or its uses.

[0038] (Stationary optical information reader) FIG. 1 is a schematic diagram illustrating a stationary optical information reader 1 according to an embodiment of the present invention during operation. In this example, multiple workpieces W are placed on the upper surface of a conveyor belt B and transported in the direction of arrow Y in FIG. 1 . The optical information reader 1 according to the embodiment is installed above and away from the workpieces W. The optical information reader 1 is a code reader configured to photograph a code attached to the workpiece W and decode the code contained in the photographed image to read information. In the example shown in FIG. 1 , the optical information reader 1 is a stationary type. When this stationary optical information reader 1 is in operation, the optical information reader 1 is fixed to a bracket or the like (not shown) to prevent movement. The stationary optical information reader 1 may also be used while being held by a robot (not shown). Alternatively, the optical information reader 1 may be configured to read the code on a stationary workpiece W. The stationary optical information reader 1 is in operation when it is performing an operation of reading the codes of the works W conveyed by the conveyor belt B in order.

[0039] In addition, a code is attached to the outer surface of each workpiece W. The code includes both a barcode and a two-dimensional code. Examples of two-dimensional codes include QR Code (registered trademark), Micro QR Code, and Data Matrix (Data code). (registered trademark) Examples of two-dimensional codes include Vericode, Aztec code, PDF417, and Maxicode. Two-dimensional codes are available in stack and matrix types, but the present invention can be applied to any two-dimensional code. The code may be attached by printing or engraving directly onto the workpiece W, or by printing on a label and then attaching it to the workpiece W; the means and method are not important.

[0040] The optical information reader 1 is connected to a computer 100 and a programmable logic controller (PLC) 101 by wired connection via signal lines 100a and 101a, respectively. However, this is not limiting, and the optical information reader 1, the computer 100, and the PLC 101 may each have a built-in communication module, and the optical information reader 1 may be connected to the computer 100 and the PLC 101 wirelessly. The PLC 101 is a control device for sequentially controlling the conveyor belt B and the optical information reader 1, and a general-purpose PLC may be used. The computer 100 may be a general-purpose or dedicated electronic computer, a portable terminal, or the like.

[0041] Furthermore, during operation, the optical information reading device 1 receives a read start trigger signal from the PLC 101 via signal line 101a, which signal specifies the timing for starting code reading. The optical information reading device 1 then captures and decodes the code based on this read start trigger signal. The decoded result is then transmitted to the PLC 101 via signal line 101a. Thus, during operation of the optical information reading device 1, the read start trigger signal is repeatedly input and the decoded result is repeatedly output via signal line 101a between the optical information reading device 1 and an external control device such as the PLC 101. Note that the input of the read start trigger signal and the output of the decoded result may be performed via signal line 101a between the optical information reading device 1 and the PLC 101, as described above, or via another signal line (not shown). For example, a sensor for detecting the arrival of a workpiece W may be directly connected to the optical information reading device 1, and the read start trigger signal may be input from the sensor to the optical information reading device 1.

[0042] 2, the optical information reader 1 is provided with a box-shaped housing 2, a polarizing filter attachment 3, an illumination unit 4, a camera 5, a display unit 6, a power connector 7, and a signal line connector 8. Furthermore, the housing 2 is provided with an indicator 9, an aimer light emitting unit (light emitting unit) 10, and operation buttons 11 and 12, and the indicator 9, the aimer light emitting unit 10, and the operation buttons 11 and 12 are also components of the optical information reader 1.

[0043] The housing 2 has a shape that is long in a predetermined direction, but the shape of the housing 2 is not limited to the shape shown in the figure. A polarizing filter attachment 3 is detachably attached to the outer surface of the front side of the housing 2. Inside this housing 2, an illumination unit 4, a camera 5, an aimer light irradiation unit 10, a processor 20, a memory unit 30, a ROM 40, a RAM 41, etc. are housed. The processor 20, the memory unit 30, the ROM 40, and the RAM 41 are also components of the optical information reading device 1.

[0044] An illumination unit 4 is provided on the front side of the housing 2. The illumination unit 4 is a part for illuminating at least the code of the workpiece W by emitting light toward the front of the optical information reading device 1. As also shown in FIG. 3, the illumination unit 4 includes a first illumination unit 4a consisting of a plurality of light-emitting diodes (LEDs), a second illumination unit 4b consisting of a plurality of light-emitting diodes, and an illumination drive unit 4c consisting of an LED driver or the like that drives the first illumination unit 4a and the second illumination unit 4b. The first illumination unit 4a and the second illumination unit 4b are driven individually by the illumination drive unit 4c so that they can be turned on and off separately. The illumination drive unit 4c is connected to the processor 20, and the illumination drive unit 4c is controlled by the processor 20. Note that one of the first illumination unit 4a and the second illumination unit 4b may be omitted.

[0045] As shown in FIG. 2, a camera 5 is provided in the center of the front side of the housing 2. The optical axis direction of the camera 5 is approximately the same as the direction of light irradiation by the illumination unit 4. The camera 5 is a part that photographs the code and generates a read image. As shown in FIG. 3, the camera 5 is equipped with an imaging element 5a that receives reflected light from the code attached to the workpiece W and illuminated by the illumination unit 4, an optical system 5b having a lens etc., and an AF module (autofocus module) 5c. Light reflected from the part of the workpiece W where the code is attached is incident on the optical system 5b, and the incident light is emitted toward the imaging element 5a and forms an image on the imaging surface of the imaging element 5a.

[0046] The imaging element 5a is an image sensor consisting of light-receiving elements such as a CCD (charge-coupled device) or a CMOS (complementary metal oxide semiconductor) that converts the image of the code obtained through the optical system 5b into an electrical signal. The imaging element 5a is connected to the processor 20, and the electrical signal converted by the imaging element 5a is input to the processor 20 as data of the read image. The AF module 5c is a mechanism that adjusts the focus by changing the position and refractive index of the focusing lens that constitutes the optical system 5b. The AF module 5c is also connected to the processor 20 and is controlled by the processor 20.

[0047] As shown in FIG. 2, a display unit 6 is provided on the side of the housing 2. The display unit 6 is, for example, an organic electroluminescence (EL) display or a liquid crystal display. The display unit 6 is connected to the processor 20 and can display, for example, the code captured by the imaging unit 5, the character string resulting from decoding the code, the read success rate, the matching level, etc. The read success rate is the average read success rate when a reading process is performed multiple times. The matching level is the reading margin that indicates the ease of reading a successfully decoded code. This can be determined from the number of error corrections that occurred during decoding, and can be expressed, for example, as a numerical value. The fewer the error corrections, the higher the matching level (reading margin), and conversely, the more the error corrections, the lower the matching level (reading margin).

[0048] A power cable (not shown) for supplying power from an external source to the optical information reader 1 is connected to the power connector 7. Signal lines 100a, 101a, etc. for communicating with the computer 100 and the PLC 101 are connected to the signal line connector 8. The signal line connector 8 can be configured, for example, by an Ethernet connector, a serial communication connector such as RS232C, a USB connector, etc.

[0049] An indicator 9 is provided on the housing 2. The indicator 9 is connected to the processor 20 and can be configured with a light-emitting element such as a light-emitting diode. The operating state of the optical information reader 1 can be notified to the outside by the lighting state of the indicator 9.

[0050] A pair of aimer light emitting units 10 are provided on the front side of the housing 2, sandwiching the camera 5. As shown in FIG. 3, the aimer light emitting unit 10 includes an aimer 10a formed of a light-emitting diode or the like, and an aimer driving unit 10b that drives the aimer 10a. The aimer 10a emits light (aimer light) toward the front of the optical information reading device 1 to indicate the shooting range and center of the field of view of the camera 5, a guide for the optical axis of the illumination unit 4, and the like. Specifically, the aimer 10a emits visible light of a color different from the ambient light (e.g., red or green) toward the shooting field of view of the camera 5, and forms a mark visible to the naked eye on the surface illuminated by the visible light. The mark may be various shapes, symbols, letters, or the like. The user can also install the optical information reading device 1 by referring to the light emitted from the aimer 10a.

[0051] As shown in FIG. 2, operation buttons 11 and 12 are provided on the side of the housing 2 to be used when setting up the optical information reader 1, etc. The operation buttons 11 and 12 include, for example, a select button and an enter button. In addition to the operation buttons 11 and 12, for example, a touch panel type operation means may be provided. The operation buttons 11 and 12 are connected to the processor 20, and the processor 20 is capable of detecting the operation states of the operation buttons 11 and 12. By operating the operation buttons 11 and 12, it is possible to select one option from multiple options displayed on the display unit 6 and to confirm the selection.

[0052] (Handheld optical information reader) In the above example, the optical information reader 1 is a stationary type, but the present invention can be applied to devices other than the stationary type optical information reader 1. Fig. 4 shows a handheld type optical information reader 1A, and the present invention can also be applied to a handheld type optical information reader 1A such as that shown in this figure.

[0053] The housing 2A of the handheld optical information reader 1A is elongated in the vertical direction. The orientation of the optical information reader 1A during use is not limited to the orientation shown in the figure, and the optical information reader 1A can be used in various orientations, but for the sake of convenience of explanation, the vertical direction of the optical information reader 1A is specified.

[0054] A display unit 6A is provided on the upper part of the housing 2A. The display unit 6A is configured in the same manner as the display unit 6 of the stationary optical information reader 1. The lower part of the housing 2A is a gripping unit 2B that the user holds during operation. The gripping unit 2B is a part that can be held in one hand by an average adult, and the shape and size can be freely set. By holding this gripping unit 2B, the optical information reader 1A can be carried around. In other words, this optical information reader 1A is a portable terminal device, and can also be called, for example, a handy terminal.

[0055] Like the stationary optical information reader 1, the handheld housing 2A also contains an illumination unit, a camera, an aimer light emitting unit, a processor, a memory unit, ROM, RAM, etc. (not shown). The optical axes of the illumination unit, the camera, and the aimer light emitting unit are directed diagonally upward from near the top end of the housing 2A. The handheld housing 2A is also provided with a buzzer (not shown).

[0056] A plurality of operation buttons 11A and a trigger key 11B are provided on or near the gripping portion 2B. The operation button 11A is similar to the operation button 11 of the stationary optical information reading device 1. When the user points the tip (top end) of the optical reading device 1A toward the workpiece W and presses the trigger key 11B, an aimer light is emitted from the tip of the optical reading device 1A. The user adjusts the orientation of the optical reading device 1A while visually observing the aimer light reflected on the surface of the workpiece W, and aligns the aimer light with the code to be read, and the code is automatically read and decoded. When reading is complete, a buzzer emits a completion notification sound.

[0057] An example of the use of the handheld optical information reading device 1A is in the picking work in a logistics warehouse. For example, when an ordered product is to be shipped from the logistics warehouse, the required product is picked from a product shelf in the warehouse. This picking work is performed by a user holding an order slip with a code written on it, who then goes to the product shelf and compares the code on the order slip with the code attached to the product or the product shelf. In this case, the handheld optical information reading device 1A alternately reads the code on the order slip and the code attached to the product or the product shelf.

[0058] (Processor configuration) The following description is common to both the stationary optical information reader 1 and the handheld optical information reader 1A, and is applicable to either device 1 or 1A unless otherwise specified. As shown in FIG. 3, the processor 20 includes a CPU core 21, a DSP core 22, and an AI chip 23. The CPU core 21, the DSP core 22, and the AI ​​chip 23 are so-called System-on-a-chip (SoC, SOC) and are mounted on a single board. Note that the DSP core 22 and the AI ​​chip 23 do not have to be SoCs, in which case they are not mounted on the same board, but this case is also within the scope of the present invention.

[0059] A high-speed RAM 41 is connected to the CPU core 21, the DSP core 22, and the AI ​​chip 23, and all of the CPU core 21, the DSP core 22, and the AI ​​chip 23 can access the RAM 41. A ROM 40 is also connected to the processor 20, and all of the CPU core 21, the DSP core 22, and the AI ​​chip 23 can access the ROM 40.

[0060] The CPU core 21 is a general-purpose processor that performs, for example, AF control, lighting control, camera control, and decoding of scanned images. The DSP core 22 performs, for example, various filter processes on scanned images. The AI ​​chip 23 is an integrated circuit dedicated to attempting to restore scanned images using a neural network, and is specialized for ultra-high-speed product-sum operations required for neural network processing.

[0061] 5, the processor 20 configures an AF control unit 20a, an imaging control unit 20b, a filter processing unit 20c, a tuning execution unit 20d, a decode processing unit 20f, an extraction unit 20g, a reduction unit 20h, an enlargement unit 20i, and an image restoration unit 20j. The AF control unit 20a, the imaging control unit 20b, the filter processing unit 20c, the tuning execution unit 20d, the decode processing unit 20f, the extraction unit 20g, the reduction unit 20h, and the enlargement unit 20i are configured by the arithmetic processing of the CPU core 21 or the DSP core 22. On the other hand, the image restoration unit 20j is configured by the AI ​​chip 23.

[0062] (AF control unit configuration) The AF control unit 20a is a unit that controls the AF module 5c shown in FIG. 3, and is configured to be able to focus the optical system 5b by conventionally known contrast AF or phase difference AF.

[0063] (Configuration of imaging control unit) The imaging control unit 20b is a unit that adjusts the gain of the camera 5, controls the amount of light from the illumination unit 4, and controls the exposure time (shutter speed) of the imaging element 5a. Here, the gain of the camera 5 refers to the amplification factor (also called magnification) when the brightness of the image output from the imaging element 5a is amplified by digital image processing. The amount of light from the illumination unit 4 can be changed by separately controlling the first illumination unit 4a and the second illumination unit 4b. The gain, the amount of light from the illumination unit 4, and the exposure time are imaging conditions for the camera 5.

[0064] (Configuration of the filter processing unit) The filter processing unit 20c is a part that applies image processing filters to the read image. The filter processing unit 20c applies a noise removal filter that removes noise contained in the image generated by the camera 5, a contrast correction filter that corrects contrast, an averaging filter, etc. The image processing filters applied by the filter processing unit 20c are not limited to a noise removal filter, a contrast correction filter, and an averaging filter, and may include other image processing filters.

[0065] The filter processing unit 20c is configured to apply an image processing filter to a read image before image restoration, which will be described later. The filter processing unit 20c is also configured to apply an image processing filter to a read image before enlargement or reduction, which will be described later. The filter processing unit 20c may be configured to apply an image processing filter to a read image after image restoration, or may be configured to apply an image processing filter to a read image after enlargement or reduction.

[0066] (Configuration of tuning execution unit) The tuning execution unit 20d shown in FIG. 5 repeatedly captures and decodes the code while varying the imaging conditions of the camera 5 and the decoding conditions of the decoding process. Based on a matching level indicating the ease of code reading calculated under each imaging and decoding condition, the tuning execution unit 20d determines optimal imaging and decoding conditions and sets the size of the code to be read. Specifically, when configuring the optical information reading device 1, 1A, the tuning execution unit 20d sets various conditions (tuning parameters) to achieve optimal decoding conditions by changing imaging conditions such as the gain of the camera 5, the light intensity and exposure time of the illumination unit 4, and the image processing conditions in the filter processing unit 20c. The image processing conditions in the filter processing unit 20c include the coefficients (strength of the image processing filter), switching between image processing filters if multiple image processing filters are used, and combining different types of image processing filters. Appropriate imaging and image processing conditions vary depending on factors such as the effect of external light on the workpiece W during transport and the color and material of the surface on which the code is attached. Therefore, the tuning execution unit 20d searches for more appropriate imaging conditions and image processing conditions, and sets the processing to be performed by the AF control unit 20a, the imaging control unit 20b, and the filter processing unit 20c.

[0067] The tuning execution unit 20d is configured to be able to acquire the size of a code included in a scanned image generated by the camera 5. To acquire the code size, the tuning execution unit 20d first searches for the code based on a feature that indicates its likelihood of being a code. Next, the tuning execution unit 20d acquires the PPC, code type, code size, etc. of the searched code. The PPC, code type, code size, etc. are included in the code parameters or code conditions, and therefore the tuning execution unit 20d is configured to be able to acquire the code parameters or code conditions of the searched code.

[0068] A two-dimensional code is composed of multiple white and black modules arranged randomly. For example, in QR Code Model 2, the number of modules ranges from 21 x 21 to 177 x 177. In the case of a stationary optical information reader 1, the number of modules and PPC are limited by reading the code to be read in advance, and the code size (pixel size) is limited to the number of modules x PPC.

[0069] When focusing on one of the modules that make up a code, the PPC is a code parameter that indicates how many pixels (picture elements) that module is made up of. After identifying one module, the tuning execution unit 20d can obtain the PPC by counting the number of pixels that make up that module.

[0070] The code type refers to the type of code, such as QR code, data matrix code, VeriCode, etc. Since each code has its own characteristics, tuning execution unit 20d can distinguish the code type, for example, based on the presence or absence of a finder pattern.

[0071] The code size can be calculated from the number of modules arranged vertically or horizontally in the code and the PPC. The tuning execution unit 20d calculates the number of modules arranged vertically or horizontally in the code, calculates the PPC, and obtains the code size by multiplying the number of modules arranged vertically or horizontally by the PPC.

[0072] (Configuration of the decoding processing unit) The decoding processing unit 20f decodes the black and white binary data. A table showing the correspondence between the coded data can be used for decoding. Furthermore, the decoding processing unit 20f checks whether the decoded result is correct according to a predetermined check method. If an error is found in the data, an error correction function is used to calculate the correct data. The error correction function differs depending on the type of code.

[0073] In this embodiment, the decoding processing unit 20f decodes the code contained in the scanned image after image restoration, which will be described later, but can also decode the code contained in the scanned image before image restoration. The decoding processing unit 20f is configured to write the decoded result obtained by decoding the code to the decoded result storage unit 30b of the storage unit 30 shown in Figure 3.

[0074] (Configuration of extraction unit) The extraction unit 20g extracts code candidate areas from the scanned image generated by the camera 5 that are likely to contain a code. The code candidate areas can be extracted based on features that indicate the likelihood of a code. In this case, the features that indicate the likelihood of a code serve as information for identifying the code. For example, the extraction unit 20g can acquire a scanned image and search for a code in the acquired scanned image based on the features that indicate the likelihood of a code. Specifically, the extraction unit 20g searches the acquired scanned image for a portion that has a predetermined or higher level of features that indicate the likelihood of a code. If a portion with a feature that indicates the likelihood of a code is found, the extraction unit 20g extracts the portion as a code candidate area. The code candidate area may include areas other than codes, but it must at least include a portion that is more likely to be a code than a predetermined level. Note that the code candidate area is merely an area where a code is likely to exist, and as a result, it may not contain a code.

[0075] The extraction unit 20g can also extract a code candidate region by using position information of the aimer light emitted from the aimer light emitting unit 10 as information for identifying a code. In this case, the extraction unit 20g identifies a portion of the read image corresponding to a mark formed by the aimer light emitting unit 10 and extracts the identified portion as a code candidate region. That is, as described above, the aimer light is used to indicate the shooting range and field of view center of the camera 5, so the mark formed by the aimer light can be aligned in advance with the field of view center of the camera 5. In particular, in the case of the handheld optical information reading device 1A, since the user aims the aimer light at the code to be read, there is a high probability that the mark formed by the aimer light will overlap with the code. That is, by aligning the portion corresponding to the mark formed by the aimer light emitting unit 10 with the field of view center of the camera 5, when the extraction unit 20g extracts a region including the field of view center of the camera 5, that region is likely to contain a code. In this case, the part corresponding to the mark formed by the aimer light emitting unit 10 does not have to be perfectly aligned with the center of the field of view of the camera 5, and may be slightly offset vertically or horizontally of the field of view. Furthermore, since the code has a predetermined size, the area extracted by the extraction unit 20g is not only the center of the field of view of the camera 5, but also an area of ​​a predetermined size that includes the center of the field of view.

[0076] The extraction unit 20g can also identify the center of the scanned image and extract the center as a code candidate area. For example, by acquiring the center of the field of view of the camera 5 in advance as information for identifying the code, the center of the field of view of the camera 5 can be identified on the scanned image. In particular, in the case of the handheld optical information reader 1A, the center identified on the scanned image corresponds to the mark formed by the aimer light emitting unit 10, and therefore the center is an area where a code is likely to exist.

[0077] The extraction unit 20g can be configured to accept a user's designation of a specific portion of the scanned image and extract the designated portion as a code candidate area. That is, when the user designates a region of any size at any position (coordinate) on the scanned image, the extraction unit 20g accepts the designation of the specific portion of the scanned image based on the coordinates and size information of the position. The extraction unit 20g extracts the accepted portion as a code candidate area. For example, in the case of a stationary optical information reading device 1, as shown in FIG. 1, a workpiece W being transported by a conveyor belt B is photographed and a code decoding process is performed. However, the workpiece W is not necessarily located in the center of the width direction of the conveyor belt B, but may be located at the edge. In this case, by designating a portion of the scanned image corresponding to the edge of the conveyor belt B, it is possible to accurately extract the region as a region where a code is likely to exist. In addition, there are cases where a code is displayed near the edge of a large workpiece W, away from the center. In such cases, by specifying a part of the scanned image that corresponds to the edge of the workpiece W, it is possible to accurately extract the area where the code is likely to be present.

[0078] (Configuration of storage unit) The storage unit 30 shown in FIG. 3 can be configured as a readable / writable storage device such as an SSD (Solid State Drive). However, the following storage units 30a, 30b, 30c, and 30d may be provided in a ROM 40 instead of the storage device. That is, this embodiment also covers a configuration in which the image data storage unit 30a, the decoded result storage unit 30b, the parameter set storage unit 30c, and the neural network storage unit 30d are included in the ROM 40. The image data storage unit 30a stores a scanned image generated by the camera 5. The decoded result storage unit 30b stores the decoded result of the code executed by the decode processing unit 20f. The parameter set storage unit 30c stores the results of tuning executed by the tuning execution unit 20d, various conditions set by the user, and various conditions set by the user. The neural network storage unit 30d stores the structure and parameters of a trained neural network, which will be described later.

[0079] (Image restoration using neural networks) The optical information reading device 1, 1A has an image restoration function that uses a neural network to restore a read image acquired by the camera 5. In this embodiment, the optical information reading device 1, 1A does not perform machine learning, but rather the optical information reading device 1, 1A stores in advance the structure and parameters of a neural network that has already completed learning, and is configured to perform image restoration using a neural network configured with the stored structure and parameters, which is different from conventional optical information reading devices.

[0080] As shown in Fig. 6, the neural network has an input layer to which input data (image data in this example) is input, an output layer that outputs output data, and an intermediate layer provided between the input layer and the output layer. For example, multiple intermediate layers can be provided, thereby making it possible to create a neural network with a multi-layer structure.

[0081] (Neural network training) First, the basic procedure for training a neural network will be described with reference to the flowchart shown in Fig. 7. Training of the neural network can be performed using a computer prepared for training purposes other than the optical information reading device 1, 1A, but it can also be performed using a general-purpose computer other than that for training. Furthermore, the neural network training method can be a conventionally known method.

[0082] After starting, in step SA1, data of a previously prepared defective image is read. A defective image is an image that has parts that are inappropriate for reading a code, and an example of such an image is shown on the left side of Figure 8. An inappropriate part is, for example, a dirty part or a colored part. Then, proceeding to step SA2, data of a previously prepared ideal image is read. An ideal image is an image that is appropriate for reading a code, and an example of such an image is shown on the right side of Figure 8. A defective image and an ideal image are paired, and multiple such pairs are prepared.

[0083] In step SA2, a loss function is calculated to determine the difference between the defective image and the ideal image. In step SA3, the neural network parameters are updated to reflect the difference determined in step SA2.

[0084] In step SA4, it is determined whether the machine learning completion condition is met. The machine learning completion condition can be set based on the difference calculated in step SA2. For example, if the difference is equal to or less than a predetermined value, it can be determined that the machine learning completion condition is met. If step SA4 returns YES and the machine learning completion condition is met, the machine learning is terminated. On the other hand, if step SA4 returns NO and the machine learning completion condition is not met, the process returns to step SA1, where another defective image is read, and then the process proceeds to step SA2, where another ideal image paired with the other defective image is read.

[0085] In this way, a trained neural network can be generated in advance by performing machine learning on multiple defective images and multiple ideal images corresponding to the multiple defective images. By storing the structure and parameters of the trained neural network in the optical information reading device 1, 1A, the trained neural network can be constructed within the optical information reading device 1, 1A. In other words, when performing machine learning, a huge number of defective images and ideal images are input into the neural network, which requires extremely high computing power. It is difficult to ensure such high computing power within the optical information reading device 1, 1A, as this may result in the device not being able to meet the demand for a compact and lightweight device. As in this example, by performing machine learning outside the optical information reading device 1, 1A and storing the structure and parameters of the resulting neural network in the optical information reading device 1, 1A, image restoration using a neural network can be achieved while realizing a compact and lightweight optical information reading device 1, 1A.

[0086] Figure 9 is a conceptual diagram of a convolutional neural network (CNN) configured as described above. "Convolution" extracts the features of the input image. This layer is composed of convolution operations similar to image processing filters. The filter weights are called kernels, and feature extraction is performed according to the kernels. Convolution layers generally have multiple different kernels, and the number of maps (D) increases according to the number of kernels. The kernel size can be, for example, 3x3 or 5x5.

[0087] "Pooling" performs a reduction process to combine the responses of each kernel. "Deconvolution" uses a deconvolution filter to reconstruct an image from the feature values. "Unpooling" performs an expansion process to sharpen the response values.

[0088] The structure and parameters of the trained neural network are stored in the neural network storage unit 30d of the storage unit 30 shown in Fig. 3. The structure of the neural network refers to the number of intermediate layers (D) provided between the input layer and the output layer, the number of image processing filters, etc. The parameters of the neural network are the parameters set in step SA3 of the flowchart shown in Fig. 7.

[0089] The neural network storage unit 30d can also store the structures and parameters of multiple neural networks with different numbers of layers or image processing filters. In this case, the structures and parameters of multiple neural networks can be stored in association with code conditions in the neural network storage unit 30d. For example, the structure and parameters of a first neural network can be associated with a first code condition corresponding to the first neural network. Since the optimal number of layers or filters of the first neural network can be determined in advance by the first code condition, particularly the first PPC, the association is performed so as to maintain this relationship. Similarly, the structure and parameters of a second neural network different from the first neural network can be associated with a second code condition different from the first code condition.

[0090] If the neural network storage unit 30d stores the structures and parameters of multiple neural networks, the tuning unit 20d operates as follows when setting the optical information reading device 1, 1A. That is, when setting the code conditions contained in the scanned image generated by the camera 5, the tuning unit 20d reads the structure and parameters of the neural network associated with the set code conditions from the neural network storage unit 30d and identifies it as the neural network to be used during operation. For example, if the tuning unit 20d sets a first code condition as the code condition contained in the scanned image, the tuning unit 20d reads the structure and parameters of the first neural network associated with the first code condition from the neural network storage unit 30d. Then, the tuning unit 20d identifies the structure and parameters of the first neural network as the neural network to be used during operation of the optical information reading device 1, 1A. This allows the scanned image to be repaired using the structure and parameters of the neural network that are optimal for the code conditions, thereby improving reading accuracy.

[0091] When training the neural network, it is also possible to train the neural network on defective images and ideal images whose pixel resolution of the modules constituting the code is within a specific range. The pixel resolution of the module can be expressed, for example, by the number of pixels (PPC) of one module constituting the code, and the neural network may be trained on only defective images and ideal images whose PPC is within a specific range.

[0092] The specific range includes pixel resolutions that provide a high level of restoration effect on scanned images, but excludes pixel resolutions that provide little or no improvement in restoration effect on scanned images, and can be set on the premise of restoring images that contain codes made up of multiple modules. This allows for an improved processing speed as a neural network structure specialized for restoring images that contain codes.

[0093] The specific range of pixel counts can be, for example, 4 PPC or more and 6 PPC or less, but it may also be less than 4 PPC or more than 6 PPC. Also, by limiting the specific range to 4 PPC or more and 6 PPC or less for machine learning, it is possible to eliminate unnecessary scale variations from the images used for learning and optimize the trained neural network structure.

[0094] (Repair attempt) As shown in Fig. 5, the processor 20 of the optical information reading device 1, 1A is provided with an image restoration unit 20j configured with an AI chip 23. The image restoration unit 20j reads out the structure and parameters of a neural network stored in the neural network storage unit 30d of the storage unit 30, and configures a neural network in the optical information reading device 1, 1A using the read-out structure and parameters of the neural network. The neural network configured in the optical information reading device 1, 1A is the same as the trained neural network trained according to the procedure shown in the flowchart of Fig. 7.

[0095] The image restoration unit 20j inputs the scanned image generated by the camera 5 into a neural network, and attempts to restore the scanned image in accordance with the structure and parameters of the neural network stored in the neural network storage unit 30d. Then, the decoding processing unit 20f executes a decoding process on the restored scanned image.

[0096] The filter processing unit 20c is configured to apply an image processing filter to the read image generated by the camera 5 before the image restoration unit 20j inputs the read image to the neural network. This allows appropriate image processing to be performed before attempting restoration using the neural network, resulting in more accurate restoration of the read image.

[0097] The image restoration unit 20j may attempt image restoration for all read images decoded by the decoding processing unit 20f, or may attempt image restoration for only some of the read images decoded by the decoding processing unit 20f. For example, the decoding processing unit 20f performs a decoding process on the read image before attempting restoration and determines whether the decoding was successful. If the decoding is successful, it means that the read image does not require restoration, and the result is output as is. On the other hand, if the decoding fails, the image restoration unit 20j attempts restoration of the read image, and the decoding processing unit 20f performs a decoding process on the restored read image.

[0098] (Input partial image) The image input by the image restoration unit 20j to the neural network may be the entire scanned image generated by the camera 5, but the larger the size of the input image, the greater the amount of calculation by the neural network, which increases the calculation load on the processor 20 and may ultimately result in a decrease in processing speed. Specifically, in an FCN (Fully Convolutional Network) type neural network as shown in Figure 9, assuming the structure is the same, the amount of calculation is proportional to the input image size, and since the input image size is proportional to the square of the code size, the amount of calculation is also proportional to the square of the code size.

[0099] On the other hand, there are virtually no cases where the code is present in the entire scanned image generated by camera 5. During normal operation, the code is present only in a small area of ​​the scanned image, and if only that area can be repaired using a neural network, the reading accuracy can be improved.

[0100] Therefore, the image input by the image restoration unit 20j to the neural network can be a part of the scanned image generated by the camera 5, i.e., a partial image including a code. Specifically, the image restoration unit 20j cuts out a partial image corresponding to the code candidate area extracted by the extraction unit 20g from the scanned image generated by the camera 5. The image restoration unit 20j inputs the cut-out partial image to the neural network, and attempts to restore the partial image according to the structure and parameters stored in the neural network storage unit 30d. The decoding processing unit 20f then performs decoding processing on the restored partial image.

[0101] As mentioned above, it is rare for a code to be present in the entire scanned image, so the size of the partial image corresponding to the code candidate area is smaller than the size of the scanned image. This reduces the size of the image input to the neural network, reducing the computational load on the processor 20 and ultimately achieving faster processing speeds. Furthermore, because the partial image corresponds to an area where a code is likely to be present, it is possible to create an image that includes the information necessary for reading, i.e., the entire code. Because this image including the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

[0102] The image restoration unit 20j can determine the input size of the partial image to the neural network based on the code size acquired by the tuning execution unit 20d. As described above, the tuning execution unit 20d can acquire the code size. The image restoration unit 20j increases the input size of the partial image to be input to the neural network by a predetermined amount compared to the code size acquired by the tuning execution unit 20d.

[0103] That is, while inputting a partial image into the neural network can reduce the computational load, if the code is rotated or the position of the code cannot be accurately detected, a portion of the code in the partial image may be missing, potentially resulting in a failure in the decoding process. In this embodiment, the input size to the neural network is not the same as the code size acquired by the tuning execution unit 20d, but is larger than the code size by a predetermined amount, thereby preventing a portion of the code from being missing from the partial image. By making the input size larger than the code size by a predetermined amount, an upper limit can be placed on the input size to the neural network, thereby preventing the computational load from becoming too heavy. The "predetermined amount" may mean, for example, that the horizontal (or vertical) length of the partial image input to the neural network is 1.5 times or more, 2.0 times or more, or 3.0 times or more the horizontal (or vertical) length of the code acquired by the tuning execution unit 20d.

[0104] When the image restoration unit 20j determines the input size of the partial image to be input to the neural network, it can determine the size based on the number of pixels in one module that makes up the code acquired by the tuning execution unit 20d and the number of modules arranged vertically or horizontally in the code.

[0105] Furthermore, the tuning execution unit 20d can set the code conditions as described above. The image restoration unit 20j can also set the size of the scanned image to be input to the neural network based on the code conditions set by the tuning execution unit 20d. For example, when the image restoration unit 20j acquires the code size from the code conditions set by the tuning execution unit 20d, it inputs a partial image that is a predetermined amount larger than the acquired code size to the neural network. If the code size is larger, the size of the partial image to be input to the neural network will be larger, and if the code size is smaller, the size of the partial image to be input to the neural network will be smaller. In other words, the size of the partial image to be input to the neural network can be changed according to the code conditions.

[0106] (Zoom in / out function) In order for a convolutional neural network like the one shown in Figure 9 to fully capture the features of the code and restore it to an image suitable for decoding, it is necessary to extract features within a range covering a certain number of modules. For example, if one feature value obtained from the neural network is a value extracted from a 6x6 module range, and an image is captured with one module consisting of 20 pixels, the neural network must be designed to calculate a feature value aggregated from a 120x120 pixel range. In other words, the wider the pixel range to be covered, the deeper the neural network hierarchy must be. For example, to aggregate feature values ​​from the above-mentioned 120x120 pixel range, six convolutional layers are required.

[0107] However, since the code is composed of a random arrangement of white and black modules, the pixel values ​​have a weak correlation over a wide range, and expanding the pixel range beyond a certain point hardly improves the restoration effect. Such features are called narrow-range features in this specification.

[0108] Furthermore, the arrangement of fixed module patterns, such as finder patterns, contained in code, or the rectangular shape of the code as a whole can be considered wide-range features, but if the module and background are simply roughly separated, repair is sufficient using only narrow-range features, without using wide-range features. Wide-range features can be defined as features that have a fixed shape over a wide range.

[0109] Therefore, when training the neural network shown in Figure 7, it is possible to use images in which the PPC of the code is within a specific range. While this can improve processing speed as a neural network structure specialized for repairing images containing codes, there is a concern that the repair effect may be reduced for scanned images in which the PPC is outside the specific range. Note that when training the neural network, it is also possible to use images in which the PPC of the code is outside the specific range.

[0110] The optical information reading devices 1 and 1A of this embodiment are provided with a function for reducing and enlarging a read image so that the PPC of the read image falls within a specific range. As shown in Fig. 5, the processor 20 is configured with a reduction unit 20h and an enlargement unit 20i. The reduction unit 20h is a part that generates a reduced read image so that the pixel resolution of the modules that make up the code in the read image generated by the camera 5 falls within the specific range, and the enlargement unit 20i is a part that generates an enlarged read image so that the pixel resolution of the modules that make up the code in the read image generated by the camera 5 falls within the specific range.

[0111] The reduction unit 20h and the enlargement unit 20i determine whether the PPC of the code in the scanned image generated by the camera 5 is outside a specific range (4 PPC to 6 PPC), and if it is within the specific range, they do not perform reduction or enlargement, but if it is outside the specific range, they perform reduction or enlargement. By performing reduction or enlargement, the PPC of the code in the partial image input to the neural network by the image restoration unit 20j falls within the specific range.

[0112] The image restoration unit 20j inputs the scanned image, enlarged or reduced to fit within a specific range, into a neural network configured with the structure and parameters stored in the neural network storage unit 30d, and attempts to restore the scanned image according to the structure and parameters. The decoding processing unit 20f then performs decoding processing on the restored scanned image.

[0113] The extraction unit 20g may be configured to search for a code in the scanned image enlarged by the enlargement unit 20i or reduced by the reduction unit 20h, and extract an area containing the searched code as a code candidate area where a code is likely to exist. In this case, the image restoration unit 20j inputs a partial image corresponding to the code candidate area extracted by the extraction unit 20g into a neural network configured with the structure and parameters stored in the neural network storage unit 30d, and attempts to restore the scanned image in accordance with the structure and parameters.

[0114] When setting up the optical information reading device 1, 1A, the tuning execution unit 20d can generate multiple read images (enlarged images) with different magnification ratios and multiple read images (reduced images) with different reduction ratios. The tuning execution unit 20d inputs the generated multiple read images (enlarged images or reduced images) into a neural network to attempt to repair each read image, and performs a decoding process on each repaired read image to determine a reading margin indicating the ease of code reading. The tuning execution unit 20d identifies the magnification or reduction ratio of the read image for which the determined reading margin is higher than a predetermined value as the magnification or reduction ratio to be used during operation. When the tuning execution unit 20d identifies the magnification or reduction ratio, during operation of the optical information reading device 1, 1A, the reduction unit 20h and the enlargement unit 20i enlarge or reduce the read image at the magnification or reduction ratio identified by the tuning execution unit 20d.

[0115] That is, for example, if the specific range has a certain width, it is conceivable that the reading margin will change if the magnification or reduction ratio is changed even within that specific range. Since the magnification or reduction ratio of the read image during operation can be specified so that the reading margin calculated by the tuning execution unit 20d is higher than a predetermined value, processing speed and reading accuracy are improved.

[0116] After determining the reading margin of each scanned image, the tuning execution unit 20d can specify the magnification or reduction ratio of the scanned image with the highest reading margin among the determined reading margins as the magnification or reduction ratio to be used during operation, thereby further improving processing speed and scanning accuracy.

[0117] Furthermore, by utilizing the ability of the reduction unit 20h and the enlargement unit 20i to reduce and enlarge the scanned image, the number of parameter sets can be increased. Multiple parameter sets with different reduction ratios by the reduction unit 20h and multiple parameter sets with different enlargement ratios by the enlargement unit 20i can be generated, and these parameter sets can be stored in the parameter set storage unit 30c of the storage unit 30. For example, if multiple reduction ratios by the reduction unit 20h can be set, such as 1 / 2, 1 / 4, and 1 / 8, the parameter set with a reduction ratio of 1 / 2, a parameter set with a reduction ratio of 1 / 4, and a parameter set with a reduction ratio of 1 / 8 can be stored in the parameter set storage unit 30c. Furthermore, if multiple enlargement ratios by the enlargement unit 20i can be set, such as 2x, 4x, and 8x, the parameter set with a magnification ratio of 2x, a parameter set with a magnification ratio of 4x, and a parameter set with a magnification ratio of 8x can be stored in the parameter set storage unit 30c. Any one of the multiple parameter sets stored in the parameter set storage unit 30c can then be applied.

[0118] (Image restoration filter) The image restoration unit 20j may be a part that executes an image restoration filter that attempts to restore the scanned image using a neural network. The image restoration filter is a filter that attempts to restore the scanned image according to the structure and parameters stored in the neural network storage unit 30d by inputting the scanned image to a neural network configured with the structure and parameters stored in the neural network storage unit 30d, and performs the same function as the image restoration described above.

[0119] When the image restoration filter is executable, a setting unit 20e (shown in FIG. 5) for setting the image restoration filter can be provided. The setting unit 20e is configured to be able to accept the setting of the image restoration filter by the user. For example, when setting up the optical information reading device 1, 1A, the setting unit 20e can be configured to generate a user interface that allows the user to select either "apply image restoration filter" or "not apply image restoration filter" and display it on the display unit 6, and accept the user's selection. Therefore, the image restoration unit 20j attempts to restore the read image by applying the image restoration filter set by the setting unit 20e to the read image.

[0120] The setting unit 20e may be included in the tuning execution unit 20d, or may be configured separately from the tuning execution unit 20d. When the setting unit 20e is included in the tuning execution unit 20d, it becomes possible to set parameters related to the image restoration filter during tuning. The parameters related to the image restoration filter can include "application of image restoration filter" and "non-application of image restoration filter." "Application of image restoration filter" means setting the image restoration filter to be executable, and "non-application of image restoration filter" means not setting the image restoration filter.

[0121] The parameters related to the image restoration filter are also information related to the image restoration filter. In this case, a parameter set including information related to the image restoration filter settings can be stored in the parameter set storage unit 30c. The parameters related to the image restoration filter include "apply image restoration filter" and "not apply image restoration filter." Therefore, the parameter sets stored in the parameter set storage unit 30c include a first parameter set that enables the image restoration filter and a second parameter set that does not set the image restoration filter. When the optical information reading device 1, 1A is operated, one of the first parameter set and the second parameter set is applied. When the first parameter set is applied, the decoding processing unit 20f performs a decoding process on a read image that has been subjected to image restoration using the image restoration filter. Meanwhile, when the second parameter set is applied, the decoding processing unit 20f performs a decoding process on a read image that has not been subjected to image restoration.

[0122] The parameter sets stored in the parameter set storage unit 30c also include items for setting code conditions included in the scanned image generated by the camera 5. The items for setting code conditions include the PPC, code type, code size, etc. acquired by the tuning execution unit 20d. For example, one parameter set may include the PPC, code type, and code size as items for setting code conditions, the gain of the camera 5, the light intensity and exposure time of the illumination unit 4 as imaging conditions, and the type of image processing filter, parameters of the image processing filter, and application items of the image restoration filter as items of the image processing filter applied by the filter processing unit 20c. The values ​​set by the tuning execution unit 20d may be used as they are, or the user may change them as desired.

[0123] (An example of tuning process steps) An example of the procedure of the tuning process performed by the tuning execution unit 20d when setting up the optical information reading device 1, 1A will be specifically described with reference to the flowchart shown in Fig. 10. In step SB1 after the start of the flowchart shown in Fig. 10, the tuning execution unit 20d controls the illumination unit 4 and camera 5 to cause the camera 5 to generate a read image, and the tuning execution unit 20d acquires the read image. At this time, the presence and type of an image processing filter to be executed before the decoding process and the decoding process parameters for applying image restoration by a neural network are set to arbitrary parameters. Next, proceeding to step SB2, the tuning execution unit 20d causes the decoding processing unit 20f to execute decoding process on the acquired read image.

[0124] After the decoding process, the process proceeds to step SB3, where the tuning execution unit 20d determines whether the decoding process of step SB2 was successful. If the determination in step SB3 is NO and the decoding process of step SB2 has failed, that is, if the code cannot be read, the process proceeds to step SB4, where the decoding process parameters are changed to other parameters, and the decoding process is executed again in step SB2. If the decoding process has failed with all of the decoding process parameters, this flow ends and the user is notified.

[0125] On the other hand, if the determination in step SB3 is YES and the decoding process in step SB2 is successful, the process proceeds to step SB5, where the tuning execution unit 20d determines code parameters (PPC, code type, code size, etc.). After the tuning execution unit 20d determines the code parameters, the structure and parameters of the neural network stored in the neural network storage unit 30d in association with the code parameters are also determined (step SB6). In step SB6, a neural network is configured using the structure and parameters read from the neural network storage unit 30d.

[0126] In step SB7, the image restoration unit 20j attempts to restore the read image by inputting the read image into the neural network configured in step SB6, and the decoding processing unit 20f executes decoding processing on the restored read image. Thereafter, the process proceeds to step SB8, where the tuning execution unit 20d evaluates the reading margin based on the decoding processing result of step SB7 and temporarily stores the evaluation result.

[0127] In step SB9, it is determined whether the decoding process has been completed for all the decoding process parameters. If the determination in step SB9 is NO, meaning that the decoding process has not been completed for all the decoding process parameters, the process proceeds to step SB10, where the decoding process parameters are changed to other parameters, and then the process proceeds to step SB7.

[0128] On the other hand, if step SB9 returns YES and the decoding process is completed for all the decoding process parameters, the process proceeds to step SB11. In step SB11, the tuning execution unit 20d selects the decoding process parameter with the highest read margin from among all the decoding process parameters, and determines the selected decoding process parameter as the parameter to be applied during operation. Note that during the tuning process, the imaging conditions and the like are also set to appropriate conditions.

[0129] (Decoding procedure before determining reduction and enlargement ratios) Next, an example of a decoding process procedure before determining the reduction ratio and enlargement ratio will be specifically described with reference to the flowchart shown in Fig. 11. The process specified in the flowchart shown in Fig. 11 can be executed in step SB2 of the flowchart shown in Fig. 10.

[0130] 11, in step SC1 after the start, the tuning execution unit 20d selects an arbitrary image processing filter from among a plurality of image processing filters. This image processing filter is a filter to be executed by the filter processing unit 20c. Then, in step SC2, the filter processing unit 20c executes the image processing filter selected in step SC1 on the scanned image.

[0131] Next, the process proceeds to step SC3, where an image pyramid is created. The image pyramid is composed of the original read image, an image obtained by reducing the original read image by half, an image obtained by reducing the original read image by a quarter, an image obtained by reducing the original read image by an eighth, and so on. The reduction of the read image is performed by a reduction unit 20h. The image pyramid can also be composed of the original read image, an image obtained by enlarging the original read image by two times, an image obtained by enlarging the original read image by four times, an image obtained by enlarging the original read image by eight times, and so on. The enlargement of the read image is performed by an enlargement unit 20i. Note that either the reduced image or the enlarged image may be omitted.

[0132] After creating the image pyramid, the process proceeds to step SC4, where an arbitrary scanned image is selected from the multiple scanned images that make up the image pyramid. The selected images may include the original scanned image that has not been reduced or enlarged. In step SC5, the extraction unit 20g extracts a code candidate area that is likely to contain a code from the scanned image selected in step SC4. Then, the process proceeds to step SC6, where the image restoration unit 20j inputs the partial image corresponding to the code candidate area extracted in step SC5 into a neural network and attempts to restore the scanned image. The size of the partial image to be input into the neural network is determined by the code conditions described above.

[0133] After the scanned image is restored, the process proceeds to step SC7, where a positioning process is performed on the outline of the code and the modules that make up the code. After the positioning process, the process proceeds to step SC8, where it is determined whether each module that makes up the code is white or black. After determining whether it is black or white, the process proceeds to step SC9, where the decoding processing unit 20f performs a decoding process on the restored scanned image. In the decoding process, for example, a method of restoring a character string from a 0-1 matrix of the modules can be used.

[0134] Thereafter, the process proceeds to step SC10, where the tuning execution unit 20d determines whether the decoding process of step SC9 was successful. If the determination in step SC10 is NO and the decoding process of step SC9 has failed, the process proceeds to step SC11, where it is determined whether all the read images constituting the image pyramid have been selected. If the determination in step SC11 is NO and all the read images constituting the image pyramid have not been selected, the process proceeds to step SC4, where another reduced read image or another enlarged read image constituting the image pyramid is selected, and the process proceeds to step SC5.

[0135] On the other hand, if the determination in step SC11 is YES and all the scanned images constituting the image pyramid have been selected, this flow ends and the conditions under which the decoding was successful are stored.

[0136] (Decoding procedure after determining reduction and enlargement ratios) Next, an example of a decoding process procedure after determining the reduction ratio and enlargement ratio will be specifically described with reference to the flowchart shown in Fig. 12. The process specified in the flowchart shown in Fig. 12 can be executed in step SB7 of the flowchart shown in Fig. 10 and when the optical information readers 1 and 1A are in operation.

[0137] In step SD1 after the start of the flowchart shown in Fig. 12, the filter processing unit 20c applies the image processing filter selected in step SC1 of the flowchart shown in Fig. 11 to the read image. At this time, if the filter processing unit 20c applies an averaging filter to the read image shown in the upper part of Fig. 13, an image after application of the image processing filter as shown in the lower part of Fig. 13 is obtained.

[0138] Then, the process proceeds to step SD2, where the scanned image is reduced or enlarged as necessary. Specifically, if the PPC of the code in the scanned image after the image processing filter is applied is within a specific range, reduction or enlargement is not performed, but if it is outside the specific range, reduction or enlargement is performed so that the PPC falls within the specific range. An example of the scanned image after reduction processing is shown in the upper part of Figure 14.

[0139] Next, in step SD3, the extraction unit 20g extracts a code candidate area that is likely to contain a code from the scanned image reduced or enlarged in step SD2. An example of an extracted code candidate area image is shown in the lower part of Fig. 14. Note that if the scanned image is not reduced or enlarged in step SD2, the extraction process is performed on the original scanned image in step SD3.

[0140] The process then proceeds to step SD4, where the image restoration unit 20j inputs the partial image corresponding to the code candidate region extracted in step SD3 into a neural network and attempts to restore the scanned image. Figure 15 shows examples of the scanned image before and after restoration. After the scanned image is restored, steps SD5 to SD7 are performed, and this flow ends. Steps SD5 to SD7 are the same as steps SC7 to SC9 in the flowchart shown in Figure 11.

[0141] (Effects of the embodiment) As described above, according to this embodiment, when the optical information reading device 1, 1A is in operation, if the read image generated by the camera 5 is input to the neural network, an attempt is made to repair the read image, and a read image after repair attempt is obtained that has been repaired to an image suitable for reading the code. A decoding process is performed on this read image after repair attempt, thereby improving reading accuracy.

[0142] The neural network used for restoration is configured with a structure and parameters obtained by prior machine learning of multiple defective images and multiple ideal images, so the optical information readers 1, 1A do not need to perform learning processing and only need to perform inference processing using the neural network, thereby reducing the calculation load on the processor 20. As a result, the handheld optical information reader 1A can be made smaller and lighter, and the processing speed of the stationary optical information reader 1 can be increased.

[0143] Furthermore, when the optical information reader 1, 1A is in operation, if a code candidate area is extracted from the scanned image, the partial image corresponding to that code candidate area is input to the neural network, and an attempt is made to repair the partial image. This reduces the size of the image input to the neural network, thereby reducing the computational load on the processor 20 and ultimately achieving faster processing speed. Furthermore, since the partial image corresponds to an area where a code is likely to exist, it is possible to create an image that includes the information necessary for reading, i.e., the entire code. Because an image including this entire code is input to the neural network, a decrease in reading accuracy is suppressed.

[0144] Furthermore, when generating the neural network structure and parameters, the neural network is trained on defective images and ideal images in which the pixel resolution of the modules that make up the code is within a specific range, thereby improving processing speed as a neural network structure specialized for repairing images that contain codes.If the pixel resolution of the modules that make up the code in the scanned image is outside the specific range, the scanned image is enlarged or reduced so that it falls within the specific range before being input to the neural network, making it possible to repair images that include codes with a wide range of pixel resolution.

[0145] (Other embodiments) 16 is a diagram illustrating an example of a processing procedure during operation of an optical information reading device 1 according to another embodiment. In this embodiment, the processor 20 includes, in addition to the image restoration unit 20j, a pre-processing unit 200, a post-processing unit 201, a first decoding unit 202, and a second decoding unit 203. The pre-processing unit 200, like the filter processing unit 20c, is a unit that applies a noise reduction filter, a contrast correction filter, an averaging filter, etc. to the read image generated by the camera 5.

[0146] The first decoding processing unit 202 is a part that executes decoding processing on the scanned image generated by the camera 5, and the decoding processing by the first decoding processing unit 202 is also referred to as the first decoding processing. When the processor 20 has a plurality of cores (N cores), the first decoding processing unit 202 is configured by cores 0 to i. The first decoding processing unit 202 may also be configured by a single core.

[0147] The second decoding unit 203 is a unit that executes decoding processing on the restored image generated by the image restoration unit 20j, and the decoding processing by the second decoding unit 203 is also referred to as second decoding processing. If the processor 20 has N cores, the second decoding unit 203 is configured by core i+1 to core N-1. The second decoding unit 203 may also be configured by a single core.

[0148] 17, when the first decoding processing unit 202 is made up of cores 0 to i, each of cores 0 to i extracts a code candidate area where a code is likely to exist from the scanned image generated by camera 5, performs positioning processing of the code outline and the modules that make up the code, and then determines whether each module that makes up the code is white or black and performs decoding processing. The processing result is output to the post-processing unit 201.

[0149] On the other hand, when the second decoding processing unit 203 is configured by core i+1 to core N-1 as shown in Fig. 17, each of core i+1 to core N-1 extracts a code candidate area that is likely to contain a code from the read image as a repair code area, and sends a trigger signal to the image repair unit 20j to execute inference processing. Although Fig. 17 shows one task being executed by two cores, one task may be executed by any one or more cores.

[0150] Specifically, when cores N-2 and N-1 constituting the second decoding processing unit 203 extract a repair code candidate area, they output a trigger signal (Trigger 1) to the image restoration unit 20j, causing the image restoration unit 20j to execute an inference process. Upon receiving the trigger signal (Trigger 1) sent from cores N-2 and N-1, the image restoration unit 20j inputs the partial image corresponding to the repair code candidate area extracted by cores N-2 and N-1 into a neural network and executes an inference process to generate a repaired image by repairing the partial image (Image Restoration 1). The repaired image generated by Image Restoration 1 is sent to cores N-2 and N-1. Based on the repaired image generated by Image Restoration 1, cores N-2 and N-1 determine grid positions indicating the positions of each cell of the code and execute a decoding process for the repaired image based on the determined grid positions.

[0151] On the other hand, when cores i+1 and i+2 constituting the second decoding processing unit 203 extract a repair code candidate area, they output a trigger signal (trigger 2) to the image restoration unit 20j, causing the image restoration unit 20j to execute an inference process. Upon receiving the trigger signal (trigger 2) sent from cores i+1 and i+2, the image restoration unit 20j inputs the partial image corresponding to the repair code candidate area extracted by cores i+1 and i+2 into a neural network and executes an inference process to generate a repaired image by repairing the partial image (image restoration 2). The repaired image generated by image restoration 2 is sent to cores i+1 and i+2. Based on the repaired image generated by image restoration 2, cores i+1 and i+2 determine grid positions indicating the position of each cell of the code and execute a decoding process for the repaired image based on the determined grid positions.

[0152] Similarly, image restoration 3 is executed in response to a trigger signal (trigger 3) sent from core N-2 and core N-1, image restoration 4 is executed in response to a trigger signal (trigger 4) sent from core i+1 and core i+2, image restoration 5 is executed in response to a trigger signal (trigger 5) sent from core N-2 and core N-1, image restoration 6 is executed in response to a trigger signal (trigger 6) sent from core i+1 and core i+2, and image restoration 7 is executed in response to a trigger signal (trigger 7) sent from core N-2 and core N-1.

[0153] As shown in this time chart, the inference processing by the image restoration unit 20j and the first decoding processing by the first decoding processing unit 202 are executed in parallel. For example, while the first decoding processing unit 202 continues to perform code region extraction, grid positioning, and decoding processing, the inference processing by the image restoration unit 20j is executed once or multiple times. Also, while the first decoding processing unit 202 continues to perform code region extraction, grid positioning, and decoding processing, the second decoding processing unit 203 extracts a repair code region, outputs a trigger signal, positions a grid, and executes second decoding processing. In this way, the image restoration unit 20j only needs to specialize in repairing a partial image corresponding to a repair code candidate region, thereby increasing the inference processing speed.

[0154] At least a portion of the time during which the second decoding unit 20j is performing the second decoding process includes a pause period during which the image restoration unit 20j does not perform inference processing. That is, as shown in FIG. 17 , after the image restoration unit 20j completes image restoration 1, the second decoding unit 203 starts the second decoding process. However, a predetermined period is provided before the image restoration unit 20j starts the next image restoration 2. This predetermined period is a pause period during which the image restoration unit 20j does not perform image restoration. This pause period is a period during which the image restoration unit 20j does not perform inference processing. By providing this pause period, the load on the image restoration unit 20j can be reduced and heat generation in the AI ​​chip 23 can be suppressed. Note that the processing time for grid positioning may be long or short depending on the image to be restored. While FIG. 17 illustrates an example in which the processing of each core is performed alternately, depending on the processing time for grid positioning, the processing of each core does not necessarily occur alternately. That is, trigger issuance and grid positioning may be performed in order starting from the core with the least processing time.

[0155] As described above, the setting unit 20e is configured to allow a user to select either "applying an image restoration filter" or "not applying an image restoration filter." Applying an image restoration filter means performing image restoration, and not applying an image restoration filter means not performing image restoration. Therefore, the setting unit 20e also corresponds to a part that sets whether or not the second decoding processing unit 203 performs the second decoding processing. Since this setting is performed by the user, the setting unit 20e is configured to be able to accept a user operation. The setting is not limited to applying or not applying an image restoration filter, but may also be, for example, applying or not applying an inference process (restoration process), or applying or not applying an AI process. The user operation may, for example, be the operation of a button or the like displayed on a user interface screen.

[0156] The optical information reading device 1 is operated after a tuning process is performed to determine in advance the settings of the imaging conditions such as exposure time and gain, and the image processing filter. For example, if the workpiece W is a label, the code is printed clearly, and if the appropriate conditions are set through the tuning process, a high reading rate can be achieved. However, for workpieces W made of metal or resin, even if the optimal conditions are set during the tuning process, reading may be difficult due to changes in the read image caused by weak ambient light, or the optical design and algorithm design may make it impossible to read due to extremely low contrast.

[0157] By performing the image restoration algorithm executed by the image restoration unit 20j in this example in parallel with the decoding process without image restoration, it is possible to improve the reading performance for workpieces W made of metal, resin, etc. while maintaining conventional reading performance, and to increase reading stability. In addition, by applying the image restoration algorithm, it becomes possible to read workpieces W that cannot be read using conventional optical designs and algorithms.

[0158] To give a specific example, in a process of reading a code printed on a sticker attached to a box in a factory, if there are no restrictions on the installation conditions of the optical information reading device 1 and optimal installation is possible, stable operation is possible even with conventional reading performance, so the image restoration filter can be disabled and takt time can be prioritized.

[0159] Also, in a certain factory, there is a process for reading codes that are clearly printed on metal, and although the optimal conditions are determined in the tuning process when setting up the reading, in actual operation there are cases where reading becomes unstable due to scratches on the metal surface, variations in the reading position and angle, changes in the brightness of the surrounding environment, etc. In such cases, applying an image restoration filter to improve the image quality makes it more resistant to variations in conditions such as those mentioned above, and improves reading performance.

[0160] In addition, some workpieces cannot be read using conventional algorithms, but in such cases, applying an image restoration filter can make them readable. For example, even if the code is severely damaged or the contrast is unclear, applying an image restoration filter can restore the contrast and scratches, making it possible to read the code.

[0161] The second decoding process executed when an image restoration filter is applied performs decoding on a restored image, and therefore often takes longer than the first decoding process, which performs decoding on a normal read image. Therefore, the first decoding processing unit 202 can execute the first decoding process in a shorter time than the time required for the second decoding process by the second decoding processing unit 203. In this example, based on this premise, the setting unit 20e is able to change the decoding timeout time, which is the time limit for the decoding process.

[0162] Specifically, when the second decoding process is set to be executed, the setting unit 20e is configured to be able to set a decoding timeout time longer than the time required to execute the second decoding process, and when the second decoding process is set not to be executed, the setting unit 20e is configured to be able to set a decoding timeout time shorter than the time required to execute the second decoding process. This allows a decoding timeout time longer than the time required to execute a typical second decoding process to be set during operation when the second decoding process is executed, thereby ensuring reliable decoding of the restored image. On the other hand, when the second decoding process is set not to be executed, only the first decoding process, which generally can be decoded in a shorter time than the second decoding process, is executed, so a short decoding timeout time corresponding to the first decoding process can be set. By changing the decoding timeout time in this manner, it is possible to set decoding timeout times corresponding to both the first decoding process and the second decoding process.

[0163] 17, the first decoding processing unit 202 and the second decoding processing unit 203 are shown as separate units, but as shown in FIG. 18, the first decoding processing unit 202 and the second decoding processing unit 203 may be combined into a decoding processing unit 210. The decoding processing unit 210 performs a second decoding processing on the restored image in parallel with a first decoding processing on the scanned image generated by the camera 5. In this way, the parts that perform the decoding processing can be arbitrarily divided or integrated on the hardware. The following description will be given using an example in which the parts that perform the decoding processing are integrated into one unit, but this is not limiting and the same applies to cases in which the parts are divided into multiple units.

[0164] When the setting unit 20e sets the second decoding process not to be executed, the decoding processing unit 210 allocates processing resources used for the second decoding process to the first decoding process, thereby speeding up the first decoding process compared to when the second decoding process is set to be executed. Processing resources include, for example, memory usage areas and cores constituting a multi-core CPU. Specifically, when the setting unit 20e sets the second decoding process not to be executed, the number of cores responsible for the first decoding process is increased compared to when the setting unit 20e sets the second decoding process to be executed. Furthermore, when the setting unit 20e sets the second decoding process not to be executed, the memory area used for the first decoding process is expanded compared to when the setting unit 20e sets the second decoding process to be executed. Increasing the number of cores and expanding the memory area may be performed together, or either one of them may be performed alone. This maximizes the performance of the multi-core CPU to speed up the first decoding process.

[0165] Prior to image restoration, the decoding processor 210 extracts code candidate regions from the scanned image that are likely to contain a code. Specifically, when using a neural network to restore an image, if the image input to the neural network is large, the computation time increases. Furthermore, from the perspective of speeding up processing, it is desirable to minimize the number of decoding attempts. To address these issues, this example employs a heat map-based search method rather than a line search-based method to ensure reliable extraction of code candidate regions, even if it takes some time. For example, the decoding processor 210 quantifies the features of the code, generates a heat map in which the magnitude of the feature is assigned to each pixel value, and extracts code candidate regions from the heat map that are likely to contain a code. A specific example is a method of extracting feature portions (e.g., finder patterns) of a 2D code from areas that are relatively hot (large feature amounts) in the heat map. If multiple feature portions are acquired, they can be prioritized and extracted, and stored in the RAM 41, the storage unit 30, etc.

[0166] After the decoding processing unit 210 extracts the code candidate region, it outputs the partial image corresponding to the extracted code candidate region to the image restoration unit 20j. The image restoration unit 20j inputs the partial image corresponding to the code candidate region extracted by the decoding processing unit 210 into a neural network and executes inference processing to generate a restored image by restoring the partial image. This reduces the calculation time.

[0167] The decoding processing unit 210 may also be configured to extract a code candidate area in the scanned image that is large enough to contain the code to be read, as determined by the tuning process described above, and that is likely to contain the code. The extracted image is then reduced or enlarged to a predetermined size before being input to the neural network, and the restored image restored by the neural network is decoded. By enlarging or reducing the extracted image to a predetermined size and inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range, thereby stably achieving the image restoration effect of the neural network. Although the code size to be read may vary, the size of the code ultimately input to the neural network can be fixed, thereby maintaining a constant balance between high processing speed and ease of decoding.

[0168] The flowcharts of the first decoding process and the second decoding process will be described in detail below with reference to FIG. 19. In step SE1 after the start, the processor 20 causes the camera 5 to generate a read image and acquires the read image. The read image acquired in step SE1 is input to the decoding processing unit 210. In step SE2, the first decoding process is executed on the read image acquired in step SE1 without performing image restoration. If the first decoding process is successful, the process proceeds to step SE4 to execute termination processing, i.e., output processing of the decoded result. If the first decoding process fails in step SE2, the process returns to step SE2 and executes the first decoding process again. When a predetermined decoding timeout period has elapsed, the first decoding process for the read image is stopped.

[0169] On the other hand, in the route proceeding from step SE1 to step SE5, the decoding processing unit 210 extracts a repair code area from the read image. Specifically, it extracts a code candidate area that is large enough to include the code to be read, as set by the tuning process, and that is likely to contain the code in the read image. If multiple repair code areas are extracted at this time, they are temporarily saved as candidate areas R1, R2, .... An example of an extracted repair code area is shown in FIG. 20. Then, it proceeds to step SE6, and the value of k is set to 1.

[0170] In step SE7, a resizing process is performed on the partial image corresponding to the candidate region Rk. The resizing process reduces or enlarges the partial image corresponding to the repair code region extracted in step SE5 to a predetermined size. This resizing process allows the size of the image to be input to the neural network, which will be described later, to be set to a predetermined size in advance. In the example shown in FIG. 20, the size of the partial image to be input to the neural network is 256 pixels x 256 pixels, but the size of the partial image is not limited to this and can be resized to any size taking into account processing speed, etc.

[0171] In step SE8, it is determined whether the black-and-white inversion setting is set to "both." In other words, image restoration is simply a mapping in mathematical terms, and it may be difficult to achieve white-to-black, black-to-white, black-to-black, and white-to-white mappings using a single neural network. Therefore, black-and-white inversion processing is performed based on the properties so that the image is printed black on a white background. Depending on how the neural network is trained, the image may be printed white on a black background. An example of black-and-white inversion processing is shown in Figure 20. Details of black-and-white inversion processing will be described later.

[0172] If the determination in step SE8 is YES and the black-and-white inversion setting is "both," the process proceeds to step SE9, where the resized image is input to the image restoration unit 20j and image restoration is performed. Thereafter, the process proceeds to step SE10, where the decoding processing unit 210 performs a second decoding process on the restored image. If the second decoding process is successful in step SE10, the process proceeds to step SE4 and executes termination processing. If the second decoding process fails in step SE10, the process proceeds to step SE11, where the resized image is subjected to black-and-white inversion processing. Thereafter, in step SE11, the image after the black-and-white inversion processing is input to the image restoration unit 20j and image restoration is performed (see FIG. 20). Next, the process proceeds to step SE13, where the decoding processing unit 210 performs a second decoding process on the restored image. If the second decoding process is successful in step SE13, the process proceeds to step SE4 and executes termination processing. If the second decoding process fails in step SE13, the process proceeds to step SE14. In step SE14, it is determined whether or not there are any more candidate regions extracted in step SE5, and if there are no more candidate regions, the process proceeds to step SE4 to execute the termination process.

[0173] On the other hand, if step SE8 returns NO and the black-and-white inversion setting is not set to "both," the process proceeds to step SE15, where it is determined whether the black-and-white inversion setting is set to "OFF." If step SE15 returns NO and the black-and-white inversion setting is not set to "OFF," the process proceeds to step SE16, where the image after the resizing process is inverted. Thereafter, in step SE17, the image after the black-and-white inversion process is input to the image restoration unit 20j, where image restoration is performed. Next, the process proceeds to step SE18, where the decoding processing unit 210 performs a second decoding process on the restored image. If the second decoding process is successful in step SE18, the process proceeds to step SE4, where termination processing is performed. If the second decoding process is unsuccessful in step SE18, the process proceeds to step SE14. In step SE14, it is determined whether or not there are still candidate areas. If there are no candidate areas, the process proceeds to step SE4, where termination processing is performed.

[0174] If it is determined in step SE14 that there are still candidate areas, the process proceeds to step SE19, where 1 is added to the value of k, and the process proceeds to step SE7. In step SE7, the value of k has increased by 1 since the previous flow, so a similar resizing process is performed on the partial image corresponding to the next candidate area Rk. Steps from SE8 onwards are as described above. In other words, if decoding is successful in either the first decoding process or the second decoding process, the decoded result is output at that point, and decoding processes are not performed on other candidate areas; instead, a scanned image of the next work W is obtained, and similar processes are performed.

[0175] The post-processing unit 201 shown in FIG. 20 performs a process of synthesizing a partial image that has not been restored after resizing with a partial image that has been restored by the image restoration unit 20j after resizing. In other words, when image restoration using a neural network is performed, the code before restoration may be visually visible to the human eye, but the image restoration may not be successful, or the cells may become round after image restoration. To address these issues, overlapping the partial image that has not been restored with the partial image that has been restored at an appropriate ratio may facilitate subsequent decoding. The overlap ratio between the partial image that has not been restored and the partial image that has been restored may be a predetermined fixed value or may be a non-fixed value. For example, it may be determined dynamically during scanning or by optimization through pre-evaluation (tuning) based on the stability of restoration by the image restoration unit 20j. For example, in pre-evaluation, the ratio may be adjustable by the user using a scale bar or the like, allowing the user to adjust the ratio while the combined image is presented to the user.

[0176] (Bank switching function) In this embodiment, the parameter set storage unit 30c is configured to store parameter sets, each of which includes a set of parameters constituting the imaging conditions of the camera 5 and parameters constituting the processing conditions in the decoding processing unit 210. A parameter set can be called a bank. A plurality of banks are provided, each storing different parameters. For example, the parameter set storage unit 30c stores a plurality of imaging conditions and code conditions, including first imaging conditions and code conditions set by the tuning execution unit 20d and second imaging conditions and code conditions, as separate parameter sets.

[0177] The optical information reading device 1 is configured to be able to switch from one parameter set including first imaging conditions and code conditions to another parameter set including second imaging conditions and code conditions, or vice versa, among the multiple parameter sets stored in the parameter set storage unit 35c. The parameter set switching can be performed by the processor 20, a user, or a switching signal from an external control device such as the PLC 101. The user can switch parameter sets by, for example, operating a parameter set switching unit incorporated in the user interface. By enabling the parameter set switching unit, the parameter set in that bank is used during operation of the optical information reading device 1, and by disabling the parameter set switching unit, the parameter set in that bank is not used during operation of the optical information reading device 1. In other words, the parameter set switching unit is used to switch from one parameter set to another.

[0178] In this example, when the decoding processing unit 210 is operating using one parameter set and the second decoding process on the restored image fails, it switches to another parameter set and performs image restoration and second decoding on a scanned image captured under different conditions. For example, when the decoding processing unit 210 detects a failure in the second decoding process, it switches to a parameter set other than the current parameter set and performs image restoration and second decoding. If it detects that the second decoding process also fails with this parameter set, it switches to a parameter set not previously used and performs image restoration and second decoding. Switching parameter sets may change not only the imaging conditions of the camera 5 but also the code size to be read. In this case, the size used to extract code candidate areas where a code is likely to exist also changes. As a result, the reduction or enlargement ratio of the image input to the neural network also changes, changing the results of image restoration.

[0179] (Black and white inversion supported) Next, we will explain how to deal with black-and-white inversion of codes with reference to Figures 21 and 22. Figure 21 is a diagram illustrating how to deal with black-and-white inversion of codes using two neural networks: one trained with a code printed in black on a white background, and the other trained with a code printed in white on a black background. Figure 21A shows a method in which a scanned image (the image on the left) is input to a neural network trained with a code printed in black on a white background and a neural network trained with a code printed in white on a black background, and the decoded results are output from each neural network. This method is called a parallel execution method. Since the scanned image is an image of a code printed in black on a white background, decoding is successful when the neural network trained with the code printed in black on a white background restores the image, but decoding fails when the neural network trained with the code printed in white on a black background restores the image.

[0180] In FIG. 21B, the scanned image (the image on the left) is first input only to a neural network trained with a code printed in white on a black background, and an attempt is made to decode the restored image. Only if the decoding process fails is the scanned image input to a neural network trained with a code printed in black on a white background to generate a restored image. This method uses a different network after a failure.

[0181] FIG. 21C shows a method for selecting a neural network to input a scanned image based on user settings. The case shown on the left is when the scanned image is set to be an image of a code printed in black on a white background. In this case, the network selection module selects a neural network trained with a code printed in black on a white background. The scanned image (the image at the left) is then input to the neural network trained with a code printed in black on a white background. On the other hand, the case shown on the right is when the scanned image is set to be an image of a code printed in white on a black background. In this case, the network selection module selects a neural network trained with a code printed in white on a black background. The scanned image (the image at the left) is then input to the neural network trained with a code printed in white on a black background.

[0182] FIG. 22 is a diagram illustrating how to deal with black-and-white inversion of a code using only one neural network. FIG. 22A shows how to generate a black-and-white inverted image by inverting the scanned image (the image on the left). An image without black-and-white inversion and an image with black-and-white inversion are input to a neural network trained with a code printed in black on a white background. Because the scanned image is an image of a code printed in white on a black background, decoding fails when using a restored image generated by a neural network trained with a code printed in black on a white background. However, decoding is successful when using the image after black-and-white inversion.

[0183] In the example shown at the top of FIG. 22B, the scanned image (the image on the left) is an image of a code printed in white on a black background, so when it is input to a neural network trained with a code printed in black on a white background, the decoding process fails. After that, when an image with black and white inversion is generated and input to a neural network trained with a code printed in black on a white background, the decoding process is successful.

[0184] In the example shown at the bottom of FIG.22B, the scanned image (the image on the left) is an image of a code printed in black on a white background, so when it is input to a neural network trained with a code printed in white on a black background, the decoding process fails. After that, when an image with black and white inversion is generated and input to a neural network trained with a code printed in white on a black background, the decoding process is successful.

[0185] FIG. 22C shows a method for determining whether to perform black-and-white inversion processing based on user settings. The case shown on the top is when the scanned image is set to be an image of a code printed in black on a white background. In this case, the black-and-white inversion processing module does not perform black-and-white inversion processing, and the image is input directly to a neural network trained with a code printed in black on a white background. On the other hand, the case shown on the bottom is when the scanned image is set to be an image of a code printed in white on a black background. In this case, the black-and-white inversion processing module performs black-and-white inversion processing. The inverted image is then input to a neural network trained with a code printed in black on a white background.

[0186] (Relationship between contrast and matching level) The matching level is used, for example, as a score during tuning, or to manage print quality / reading performance during operation. For example, the matching level is defined as a positive integer value between 0 and 100, and a level of 50 or higher can be designed to achieve a 100% read rate when a read rate test is conducted. Also, a matching level of 0 means that the code cannot be read, and a matching level of 1 is the lowest value at which the code can be read.

[0187] 23 is a graph showing the relationship between contrast and matching level, where the solid line shows the relationship between contrast and matching level of an image (restored image) restored by the image restoration unit 20j, and the dashed line shows the relationship between contrast and matching level of an image (read image) that has not been restored by the image restoration unit 20j. As is clear from this graph, the matching level of the restored image is generally higher than that of the read image, but when the image quality deteriorates beyond a certain level, the matching level drops sharply.

[0188] (Matching level calculation process) A matching level is set for each algorithm, and typically one algorithm is used for each code type. However, in this embodiment, an image restoration function is implemented, and two algorithms (one with image restoration and one without image restoration) are executed for each code type. The result of the algorithm that first successfully decodes the image is output. Therefore, if the two algorithms alternately succeed in decoding, the matching level value may become unstable. Furthermore, since image restoration makes the image easier to read, conventional methods of calculating the matching level tend to result in a higher matching level than existing methods. In response to these issues, this embodiment calculates a weighted sum of the matching levels of the two algorithms and adjusts the matching level value. A specific example will be described below based on the flowchart shown in FIG. 24.

[0189] In step SF1 after starting, the read image is input to the decoding processing unit 210. In step SF2, the decoding processing unit 210 executes the first decoding process without image restoration. In step SF3, a matching level A (MLA) is calculated by the first decoding process. Meanwhile, in step SF4, the read image is input to the image restoration unit 20j and image restoration is executed. Thereafter, the process proceeds to step SF5, where the decoding processing unit 210 executes the second decoding process on the restored image. In step SF6, a matching level B (MLB) is calculated by the second decoding process. Note that if decoding is not successful in step SF2, MLA=0, and if decoding is not successful in step SF5, MLB=0.

[0190] In step SF7, the MLA and MLB values ​​are adjusted using, for example, an averaging function, etc. Furthermore, regardless of whether decoding is successful in step SF2 or SF5, a process of weighting the MLA value is performed.

[0191] (Example of user interface) FIG. 25 is a diagram showing an example of a user interface 300 displayed on the display unit 6. The processor 20 generates the user interface 300 and displays it on the display unit 6. The user interface 300 is provided with a first image display area 301 in which an image currently captured by the camera 5 (a live view image) is displayed, a second image display area 302 in which a read image is displayed, and a third display area 303 in which a restored image restored by the image restoration unit 20j is displayed. This allows the read image and the restored image to be presented to the user in a form that allows them to compare them. Furthermore, the user can understand what the image to be decoded or what the image after the decoding process is like is like.

[0192] (Example of an image sensor with an AI chip) FIG. 26 shows an example in which an imaging element 5a with an AI chip is used, and the imaging element 5a of the camera 5 can be configured as shown in each figure. FIG. 26A shows a packaged imaging element 5a equipped with an AI chip that performs image restoration. After the scanned image is restored, it is output to the processor 20. FIG. 26B shows a packaged imaging element 5a equipped with an AI chip that extracts a repair code area and performs image restoration. After extracting a repair code area from the scanned image, it repairs a partial image corresponding to that area and outputs it to the processor 20. FIG. 26C shows a packaged imaging element 5a equipped with an AI chip that extracts a repair code area. After extracting a repair code area from the scanned image, it outputs a partial image corresponding to that area to the processor 20.

[0193] In the example shown in FIG. 26A, the scanned image acquired by the image sensor 5a may be output to the processor 20, and the image restoration may be performed by an AI chip, and the restored image may be output to the processor 20. The same applies to the examples shown in FIGS. 26B and 26C.

[0194] (Polarizing filter attachment 3) In this embodiment, as shown in Fig. 2, the device includes a first illumination unit 4a and a second illumination unit 4b, each consisting of a plurality of light-emitting diodes, as light sources that generate illumination light for illuminating at least the code. The polarizing filter attachment 3 includes a first polarizing plate 3a that passes light of a first polarization component of the light generated by the light-emitting diodes that constitute the second illumination unit 4b, and a second polarizing plate 3b that passes light of a second polarization component that is approximately perpendicular to the first polarization component. The first polarizing plate 3a and the second polarizing plate 3b can be provided in the shaded areas in Fig. 2. For example, the first polarizing plate 3a is arranged to cover the second illumination unit 4b, and the second polarizing plate 3b is arranged to cover the optical system 5b of the camera 5 from the light incident side.

[0195] Therefore, the camera 5 receives light that passes through the first polarizer 3a and is reflected from the surface (code) of the workpiece W via the second polarizer 3b. This removes the specular reflection component of the workpiece W, and the camera 5 generates a read image with lower contrast than when the light does not pass through the first polarizer 3a and the second polarizer 3b. This low-contrast read image is acquired by the processor 20. The processor 20 inputs the low-contrast read image into a neural network, converting it into a restored image with higher contrast than before input, and performs a decoding process on the restored image.

[0196] In other words, by using polarizing plates 3a and 3b, the specular reflection component of the workpiece W is removed, making it possible to obtain a read image in which the influence of the specular reflection component is reduced. For example, polarizing plates 3a and 3b are suitable for images with a large specular reflection component, such as when capturing an image of a metal workpiece. However, the read image may become dark and the contrast may decrease due to a decrease in the amount of light. In such cases, reading performance can be improved by converting the image into a high-contrast restored image using neural network inference processing.

[0197] The above-described embodiments are merely examples in all respects and should not be construed as limiting. Furthermore, all modifications and variations within the scope of the claims are within the scope of the present invention. [Industrial Applicability]

[0198] As described above, the optical information reading device according to the present invention can be used, for example, to read a code attached to a workpiece. [Explanation of symbols]

[0199] 1. Stationary optical information reader 1A Handheld optical information reader 2. Housing 2A Handheld Housing 2B Gripping part 5. Camera 20 processors 20c Filter processing section 20d Tuning Execution Department 20e Settings 20f Decode processing section 20g extraction part 20h Reduced part 20i Enlarged section 20j Image restoration section 30 Storage section 30d Neural network memory section double work

Claims

1. An optical information reading device that reads a code attached to a workpiece, a camera that photographs the code and generates a scanned image; a memory unit that stores the structure and parameters of a neural network that has been generated in advance by machine learning the defective images and ideal images, each of which has a pixel resolution that represents the number of pixels of one module that constitutes the code, among a plurality of defective images having portions that are unsuitable for reading the code and a plurality of ideal images that respectively correspond to the plurality of defective images; a processor that inputs the scanned image generated by the camera, which has been enlarged or reduced so that the pixel resolution of modules constituting the code in the scanned image approaches the specific range from outside the specific range, into a neural network configured with the structure and parameters stored in the storage unit, thereby attempting to repair the scanned image in accordance with the structure and parameters stored in the storage unit, and executing a decoding process on the repaired scanned image, An optical information reading device characterized in that the neural network generated by the machine learning has a higher repair effect on codes whose pixel resolution is within the specific range than on codes whose pixel resolution is outside the specific range.

2. 2. The optical information reading device according to claim 1, The structure and parameters of the neural network are generated in advance by machine learning the defective image and the ideal image, each of which has a module comprising a code with a pixel number within a specific range; An optical information reading device in which the processor inputs the scanned image generated by the camera, enlarged or reduced so that the number of pixels of the modules making up the code in the scanned image falls within the specific range, into the neural network.

3. 3. The optical information reading device according to claim 1, The optical information reading device further comprises a filter processing unit that applies an image processing filter to the read image generated by the camera before enlarging or reducing the read image.

4. 4. The optical information reading device according to claim 1, a tuning execution unit that, when setting the optical information reading device, generates a plurality of read images with different magnification ratios or a plurality of read images with different reduction ratios, inputs the generated plurality of read images into the neural network to attempt to repair each read image, executes a decoding process on each repaired read image to obtain a reading margin, and specifies the magnification ratio or reduction ratio of the read image for which the obtained reading margin is higher than a predetermined value as the magnification ratio or reduction ratio to be used during operation; The processor enlarges or reduces the read image at the enlargement ratio or reduction ratio specified by the tuning execution unit when the optical information reading device is in operation.

5. 5. The optical information reading device according to claim 4, The tuning execution unit specifies the enlargement or reduction ratio of the read image with the highest reading margin among the reading margins of the read images as the enlargement or reduction ratio to be used during operation of the optical information reading device.

6. 4. The optical information reading device according to claim 1, the storage unit stores structures and parameters of a plurality of neural networks having different numbers of layers or different numbers of image processing filters in association with code conditions; a tuning execution unit that sets code conditions included in the scanned image generated by the camera when setting the optical information reader, reads out the structure and parameters of a neural network associated with the set code conditions from the storage unit, and identifies the neural network to be used during operation; The processor attempts to restore the read image using the neural network specified by the tuning execution unit during operation of the optical information reading device.

7. 7. The optical information reading device according to claim 4, The tuning execution unit sets code conditions to be included in the read image generated by the camera, acquires the number of pixels of the module that meets the set code conditions, and if the acquired number of pixels is within the specific range, prevents the processor from enlarging or reducing the code conditions.

8. 5. The optical information reading device according to claim 4, the storage unit is configured to store a plurality of parameter sets each having different enlargement ratios or a plurality of parameter sets each having different reduction ratios, The processor executes a decoding process by applying any one parameter set from among a plurality of parameter sets stored in the storage unit.

9. 7. The optical information reading device according to claim 1, The processor searches for a code in the enlarged or reduced read image based on information for identifying the code, extracts an area containing the searched code as a code candidate area where a code is likely to exist, inputs a partial image corresponding to the extracted code candidate area into a neural network configured with the structure and parameters stored in the storage unit, thereby attempting to repair the partial image in accordance with the structure and parameters stored in the storage unit, and performs a decoding process on the repaired partial image.

10. 10. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a first decoding processing unit that executes a first decoding process on the scanned image generated by the camera; a second decoding processing unit that executes a second decoding process on the restored image generated by the image restoration unit, the second decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image, and sends a trigger signal to the image restoration unit to cause it to execute an inference process; when receiving the trigger signal sent from the second decoding processing unit, the image restoration unit inputs a partial image corresponding to the code candidate region extracted by the second decoding processing unit into the neural network, and executes an inference process to generate a restored image by restoring the partial image; The second decoding processing unit determines grid positions indicating the positions of each cell of the code based on the restored image, and performs decoding processing of the restored image based on the determined grid positions.

11. 11. The optical information reading device according to claim 10, An optical information reading device in which the inference processing by the image restoration unit and the first decoding processing by the first decoding processing unit are executed in parallel, and at least a portion of the period during which the second decoding processing unit is executing the second decoding processing has a pause period during which the inference processing by the image restoration unit is not executed.

12. 12. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a first decoding processing unit that executes a first decoding process on the scanned image generated by the camera; a second decoding processing unit that executes a second decoding process on the restored image generated by the image restoration unit; a setting unit that sets whether or not the second decoding process is to be executed, the first decoding processing unit is capable of executing the first decoding process in a shorter time than a time required for the second decoding process; The setting unit is configured to set a decode timeout period longer than the time required to execute the second decoding process when the second decoding process is set to be executed, and to set a decode timeout period shorter than the time required to execute the second decoding process when the second decoding process is set not to be executed.

13. 10. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a decoding processing unit that performs a first decoding process on the read image generated by the camera, and a second decoding process on the restored image generated by the image restoration unit in parallel with the first decoding process; a setting unit that sets whether or not the second decoding process is to be executed, When the setting unit is set not to execute the second decoding process, the decoding processing unit allocates the processing resources used for the second decoding process to the first decoding process, thereby speeding up the first decoding process compared to when the setting unit is set to execute the second decoding process.

14. 10. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a decoding processing unit that executes a decoding process on the restored image generated by the image restoration unit, the decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image, the image restoration unit inputs a partial image corresponding to the code candidate region extracted by the decoding processing unit into the neural network, and executes an inference process to generate a restored image by restoring the partial image; The decoding processing unit determines grid positions indicating the positions of each cell of the code based on the restored image, and decodes the restored image based on the determined grid positions.

15. 10. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a decoding processing unit that executes a decoding process on the restored image generated by the image restoration unit; a tuning execution unit that repeats the image capturing and decoding processes by changing the image capturing conditions of the camera and the decoding conditions of the decoding process, and determines optimal image capturing and decoding conditions based on a matching level that indicates the ease of reading the code calculated under each image capturing and decoding condition, and executes a tuning process that sets the size of the code to be read, The decoding processing unit An optical information reading device that extracts a code candidate area from the read image that is large enough to include the code to be read, as set by the tuning process, and that is likely to contain a code, reduces or enlarges the extracted image to a predetermined size, and then inputs it into the neural network, and decodes the restored image restored by the neural network.

16. 10. The optical information reading device according to claim 1, a plurality of light sources that generate illumination light for illuminating at least the code; a first polarizing plate that transmits light of a first polarization component out of the light generated by the light source; a second polarizing plate that transmits light of a second polarized component that is substantially orthogonal to the first polarized component; The camera receives light that passes through the first polarizing plate and is reflected from the code via the second polarizing plate, and removes the regular reflection component of the work, thereby generating a read image with lower contrast than when the first polarizing plate and the second polarizing plate are not used, The processor inputs the low-contrast read image into the neural network, converts it into a restored image with higher contrast than before input, and performs a decoding process on the restored image.

17. 10. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a decoding processing unit that executes a decoding process on the restored image generated by the image restoration unit; a tuning execution unit that is capable of changing the imaging conditions of the camera, repeating imaging and decoding processes, and setting a plurality of imaging conditions and code conditions for the code to be read based on a matching level that indicates the ease of reading the code calculated under each imaging condition, the storage unit stores a plurality of imaging conditions and code conditions including first imaging conditions and code conditions and second imaging conditions and code conditions set by the tuning execution unit; The decoding processing unit inputs a read image generated under the first imaging condition and code condition out of the multiple reading conditions stored in the memory unit into the neural network to generate a repaired image, and if the decoding process of the repaired image fails, inputs a read image generated under the second imaging condition and code condition into the neural network to generate a repaired image, and performs decoding of the repaired image.

18. 18. An optical information reading device according to any one of claims 1 to 17, the camera generates a first scanned image including a first code and a second scanned image including a second code; the processor enlarges or reduces the first read image and the second read image, in which the pixel resolution of both the first code and the second code is outside the specific range, and then inputs the enlarged or reduced image to the neural network; An optical information reading device characterized in that, when the pixel resolution of the first code falls within the specific range after the enlargement or reduction and the pixel resolution of the second code remains outside the specific range, the repair effect of the neural network on the first code is higher than the repair effect on the second code.

Images