Optical information reading device

Abstract

This neural network structure, specifically designed for repairing images containing codes, improves processing speed while enabling the repair of images with codes across a wide range of pixel resolutions. [Solution] The processor attempts to repair the read image by inputting the read image, which has been enlarged or reduced so that the pixel resolution of the modules constituting the code is within a specific range, into a neural network composed of structures and parameters stored in the memory unit, and then performs a decoding process on the repaired read image.

Description

Technical Field

[0001] The present invention relates to an optical information reading device that optically reads information.

Background Art

[0002] In recent years, so-called traceability, which enables tracking of, for example, the distribution route of goods from the manufacturing stage to the consumption stage or the disposal stage, has been regarded as important, and code readers for this traceability purpose have become widespread. In addition to traceability, code readers are also used in various fields.

[0003] Generally, a code reader is configured to be able to photograph a code such as a barcode or a two-dimensional code attached to a workpiece with a camera, cut out the code included in the obtained image by image processing, binarize it, and perform a decoding process to read information. Since it is a device that optically reads information, it is also called an optical information reading device.

[0004] As this type of optical information reading device, for example, as disclosed in Patent Document 1, there is known one provided with a machine learning device that learns a model structure showing the relationship between an image of a code acquired by a visual sensor of a robot and an image of an ideal code. Patent Document 1 describes that by applying the learning result by the machine learning device to the image of the code acquired by the visual sensor during operation, the image is restored to an image suitable for reading.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] By the way, in order to fully capture the characteristics of the code and restore it to an image suitable for reading, it is necessary to extract features within a range that covers a certain number of modules. For example, if one feature value obtained from a neural network is a value extracted from a range of 6x6 modules, and the image is taken with one module of 20 pixels, then the neural network must be designed to calculate aggregated feature values ​​from a range of 120x120 pixels.

[0007] In other words, the wider the pixel range you want to cover, the deeper the neural network layers must be. For example, to aggregate feature values ​​from the aforementioned 120x120 pixel range, six convolutional layers would be required.

[0008] However, since the code is composed of randomly arranged white and black modules, the relationships between pixel values ​​over a wide range are weak, and the repair effect hardly improves even if the covered pixel range is expanded beyond a certain point.

[0009] This invention has been made in view of the above, and its objective is to enable the repair of images containing codes with a wide range of pixel resolutions, while improving processing speed as a neural network structure specialized for repairing images containing codes. [Means for solving the problem]

[0010] To achieve the above objective, one aspect of this disclosure may be based on an optical information reading device for reading a code attached to a workpiece. The optical information reading device includes a camera that photographs the code and generates a read image; a storage unit that stores the structure and parameters of a neural network that has been pre-generated by machine learning the defective images and ideal images, which are among a plurality of defective images having portions unsuitable for reading the code and a plurality of ideal images corresponding to each of the plurality of defective images, wherein the pixel resolution of the modules constituting the code is within a specific range; and a processor that inputs the read image, which has been enlarged or reduced so that the pixel resolution of the modules constituting the code in the read image generated by the camera is within the specific range, into a neural network composed of the structure and parameters stored in the storage unit, thereby attempting to repair the read image according to the structure and parameters stored in the storage unit, and performing a decoding process on the repaired read image.

[0011] This configuration uses machine learning to generate the neural network structure and parameters by studying defective and ideal images where the pixel resolution of the modules constituting the code is within a specific range. This specific range includes pixel resolutions that yield high repair results and excludes pixel resolutions that show little improvement in repair results. It can be set with the assumption of repairing images containing code composed of multiple modules. This improves processing speed by creating a neural network structure specifically designed for repairing images containing code.

[0012] Furthermore, if the pixel resolution of the modules constituting the code in the image read by the camera is outside the specified range, the read image is enlarged or reduced to bring it within the specified range before being input to the neural network, thus enabling the repair of images containing codes with a wide range of pixel resolutions.

[0013] While the arrangement of fixed module patterns, such as finder patterns, within the code, and the fact that the code as a whole is rectangular, can be broad-range features, if the goal is simply to roughly separate modules from the background, it is possible to sufficiently restore the code using only narrow-range features without needing broad-range features.

[0014] In other aspects of the present disclosure, the structure and parameters of the neural network are pre-generated by machine learning the defective image and the ideal image in which the number of pixels of the modules constituting the code is within a specific range, and the processor can input the read image, which has been enlarged or reduced so that the number of pixels of the modules constituting the code in the read image generated by the camera is within the specific range, into the neural network.

[0015] This configuration allows for the creation of a neural network structure that is more specialized for repairing images containing code, by limiting the number of pixels to a specific range. The specific range for the number of pixels can be, for example, 4 PPC or more and 6 PPC or less, but it may also be less than 4 PPC or more than 6 PPC. PPC is a unit that indicates how many pixels (picture elements) one module has.

[0016] Furthermore, by limiting the machine learning process to a specific range of 4 PPCs or more and 6 PPCs or less, it is possible to eliminate unnecessary scale variations from the images used for training, thereby optimizing the neural network structure.

[0017] In other aspects of this disclosure, the system further includes a filter processing unit that applies an image processing filter to the image read by the camera before enlarging or reducing the image. This allows appropriate image processing to be performed before enlarging or reducing the image, resulting in better and more accurate restoration of the image. Examples of image processing filters include noise reduction filters.

[0018] In another aspect of the present disclosure, the optical information reading device further includes a tuning execution unit that, when the optical information reading device is set up, generates a plurality of reading images with different magnifications or a plurality of reading images with different reduction ratios, inputs the generated plurality of reading images into the neural network to attempt to repair each reading image, performs a decoding process on each repaired reading image to determine the reading margin, and identifies the magnification or reduction ratio of the reading image whose obtained reading margin is higher than a predetermined value as the magnification or reduction ratio to be used during operation. The processor can enlarge or reduce the reading image to the magnification or reduction ratio identified by the tuning execution unit when the optical information reading device is in operation.

[0019] In other words, if the specified range has a certain width, changing the magnification or reduction ratio even within that specified range may change the read margin. In this configuration, the magnification or reduction ratio of the read image during operation can be specified so that the read margin is higher than predetermined, thereby improving processing speed and reading accuracy.

[0020] In other aspects of this disclosure, the tuning execution unit identifies the magnification or reduction ratio of the image with the highest reading margin among the reading margins of each read image as the magnification or reduction ratio to be used during operation, thereby further improving processing speed and reading accuracy.

[0021] In other aspects of the present disclosure, the storage unit stores the structures and parameters of a plurality of neural networks with different numbers of layers or image processing filters, associated with coding conditions. The optical information reading device further includes a tuning execution unit that sets coding conditions included in the image read by the camera, reads the structures and parameters of the neural networks associated with the set coding conditions from the storage unit, and identifies them as neural networks to be used during operation. The processor can attempt to repair the read image using the neural network identified by the tuning execution unit during operation of the optical information reading device.

[0022] That is, since the number of layers or filters of the neural network can be pre-specified as the optimal number according to the code conditions, particularly the PPC, these can be associated with each other. And if the code conditions are specified when setting up the optical information reading device, the structure and parameters of the neural network associated with the code conditions can also be specified. Therefore, since repair can be attempted with the structure and parameters of the neural network optimal for the code conditions, the reading accuracy can be improved.

[0023] In another aspect of the present disclosure, the tuning execution unit sets the code conditions included in the reading image generated by the camera, obtains the number of pixels of the module that is the set code conditions, and if the obtained number of pixels is within the specific range, the processor is prohibited from performing the enlargement or reduction, so that enlargement or reduction can be performed only when the number of pixels of the module is outside the specific range.

[0024] In another aspect of the present disclosure, the storage unit is configured to store a plurality of parameter sets with different enlargement ratios by the processor or a plurality of parameter sets with different reduction ratios by the processor. The processor can execute the decoding process by applying any one of the plurality of parameter sets stored in the storage unit. Thereby, for example, when the number of pixels of the module of the code changes, the optical information reading device can be operated by applying the parameters of the enlargement ratio or reduction ratio corresponding to the number of pixels.

[0025] In another aspect of the present disclosure, the processor searches for a code in the enlarged or reduced read image based on information for identifying the code, extracts regions containing the searched code as code candidate regions where the code is likely to exist, inputs a partial image corresponding to the extracted code candidate region into a neural network configured with structures and parameters stored in the memory, attempts to repair the partial image according to the structures and parameters stored in the memory, and performs a decoding process on the repaired partial image.

[0026] This configuration allows for a reduction in the size of the image input to the neural network. This reduces the computational load on the processor, resulting in faster processing speeds. Furthermore, since the partial image corresponds to a region where the code is likely to exist, it is possible to create an image that includes all the information necessary for reading, i.e., the entire code. By inputting this image containing the entire code into the neural network, the decrease in reading accuracy is suppressed.

[0027] In another aspect of the present disclosure, there is further provided an image restoration unit that inputs a captured image into a neural network and executes an inference process to generate a restored image obtained by restoring the captured image, a first decoding process unit that executes a first decoding process on the captured image generated by a camera, and a second decoding process unit that executes a second decoding process on the restored image generated by the image restoration unit. The second decoding process unit extracts a code candidate region from the captured image and sends a trigger signal to the image restoration unit to trigger the execution of the inference process. When the image restoration unit receives the trigger signal sent from the second decoding process unit, it inputs a partial image corresponding to the code candidate region into the neural network and executes an inference process to generate a restored image obtained by restoring the partial image. The second decoding process unit can determine grid positions indicating each cell position of the code based on the restored image and execute decoding of the restored image based on the determined grid positions. That is, the first decoding process unit executes decoding of a normal captured image that has not been subjected to inference processing, the image restoration unit executes generation of a restored image by inference processing, and the second decoding process unit executes decoding of the restored image restored by inference processing, thereby constructing a processing circuit suitable for each and achieving a higher processing speed. In addition, since the image restoration unit only needs to restore the partial image corresponding to the code candidate region, it is possible to further increase the processing speed.

[0028] In another aspect of the present disclosure, by executing the inference process by the image restoration unit and the first decoding process by the first decoding process unit in parallel, it is possible to achieve a higher processing speed. In addition, since at least a part of the period during which the second decoding process unit executes the second decoding process has a pause period during which the inference process by the image restoration unit is not executed, it is possible to reduce the load on the image restoration unit and suppress heat generation. In this case, the second decoding process unit may be composed of a plurality of cores.

[0029] In other aspects of this disclosure, the first decoding processing unit can perform the first decoding process in a shorter time than the time required for the second decoding process. Furthermore, a setting unit can be used to switch whether or not to perform the second decoding process, i.e., whether or not to perform the decoding of the restored image. Since the second decoding process performs the decoding of a restored image, it takes longer than the first decoding process, which performs the decoding of a normal read image. Given this, if the setting is to perform the second decoding process, a decoding timeout period (decoding limit) longer than the time required to perform the second decoding process can be set. On the other hand, if the setting is not to perform the second decoding process, a decoding timeout period shorter than the time required to perform the second decoding process can be set. Therefore, if the decoding of the restored image exceeds the decoding timeout period, the decoding process can quickly proceed to the decoding of the next image captured.

[0030] A decoding processing unit according to another aspect of this disclosure can perform a second decoding process on a restored image in parallel with a first decoding process on a read image generated by a camera, thereby increasing processing speed even when read images and restored images are mixed. If the setting unit is configured not to perform the second decoding process, this decoding processing unit can allocate the processing resources used for the second decoding process to the first decoding process. Processing resources include, for example, the memory usage area or the cores that make up a multi-core CPU. If memory areas or cores are pre-configured as processing resources used for the second decoding process, and the system is configured not to perform the second decoding process, then those memory areas or cores can be used for the first decoding process, thus further increasing the speed of the first decoding process.

[0031] A decoding processing unit according to another aspect of this disclosure extracts code candidate regions from a read image that are highly likely to contain a code. An image restoration unit inputs a partial image corresponding to the code candidate region into a neural network and generates a restored image by restoring the partial image. A decoding processing unit determines grid positions based on the restored image generated by the image restoration unit and performs decoding of the restored image based on the determined grid positions. In other words, since generating a restored image often takes time, the time required to generate the restored image can be shortened by extracting code candidate regions based on, for example, features that indicate code-likeness before generating the restored image. Furthermore, since the grid position can be determined with high accuracy based on the restored image, reading performance is improved.

[0032] In other aspects of this disclosure, a tuning process is performed to determine optimal imaging and decoding conditions and to set the size of the code to be read. Based on the code size set in the tuning process, the decoding processing unit can determine the size of an extracted image containing the code candidate region from the read image. By enlarging or reducing the extracted image to a predetermined size and inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range, and the image restoration effect by the neural network can be stably obtained. Furthermore, although the code size of the code to be read may vary, the size of the code ultimately input to the neural network can be fixed, so a balance between processing speed and ease of decoding can be maintained at a certain level or higher.

[0033] A camera according to another aspect of this disclosure receives light that has passed through a first polarizing plate and reflected from the code via a second polarizing plate, generating a lower-contrast read image compared to the case without the first and second polarizing plates. By using polarizing plates, the specular reflection component of the workpiece is removed, so a read image with reduced influence of specular reflection can be obtained. Polarizing plates are suitable for images with a large specular reflection component, such as when imaging a metal workpiece, but the reduced light intensity can cause the read image to become darker and the contrast to decrease. In such cases, reading performance can be improved by converting it into a high-contrast restored image through neural network inference processing.

[0034] In other aspects of this disclosure, a tuning execution unit can set a plurality of imaging conditions and coding conditions, including a first imaging condition and coding condition and a second imaging condition and coding condition. The decoding processing unit inputs the read image generated with the first imaging condition and coding condition into a neural network to generate a restored image, and if the decoding of the restored image fails, it can input the read image generated with a second imaging condition and coding condition different from the first imaging condition and coding condition into the neural network to generate a restored image and perform decoding of the restored image. [Effects of the Invention]

[0035] As explained above, by machine learning a neural network that has been pre-generated by training it with defective images and ideal images in which the pixel resolution of the modules constituting the code is within a specific range, a scanned image that has been enlarged or reduced so that the pixel resolution of the modules constituting the code in the scanned image is within that specific range can be input, and the neural network can attempt to repair the scanned image. This allows for the repair of scanned images containing code with a wide range of pixel resolutions while improving processing speed by creating a neural network structure specialized for repairing scanned images containing code. [Brief explanation of the drawing]

[0036] [Figure 1]This diagram illustrates the operation of a stationary optical information reading device. [Figure 2] This is a perspective view of a stationary optical information reading device. [Figure 3] This is a block diagram of an optical information reading device. [Figure 4] This is a perspective view of a handheld optical information reading device. [Figure 5] This diagram illustrates the various components that make up the processor. [Figure 6] This is a conceptual diagram of a neural network. [Figure 7] This flowchart shows an example of the basic steps involved in training a neural network. [Figure 8] The following are pairs of images, one showing a bad image and the other an ideal image, where (A) is an example of a bad image and (B) is an example of an ideal image. [Figure 9] This is a conceptual diagram of a convolutional neural network used for image transformation. [Figure 10] This flowchart shows an example of the tuning procedure performed when setting up an optical information reading device. [Figure 11] This flowchart shows an example of the decoding process procedure before determining the reduction and expansion ratios. [Figure 12] This flowchart shows an example of the decoding process after determining the reduction and enlargement ratios. [Figure 13] (A) shows an example of a read image generated by the camera, and (B) shows an example of an image after applying an image processing filter. [Figure 14] (A) shows an example of an image after reduction processing, and (B) shows an example of an image after code region extraction. [Figure 15] This figure shows an example of image restoration using a neural network. [Figure 16] This figure shows an example of a decoding process procedure according to another embodiment. [Figure 17] This is a time chart of the decoding process performed by the first and second decoding processing units. [Figure 18]This is a time chart of the decoding process performed by the decoding unit. [Figure 19] This flowchart shows an example of the first and second decoding processes. [Figure 20] This figure shows example images for each process. [Figure 21] This diagram illustrates how to handle the black-and-white inversion of code using multiple neural networks. [Figure 22] This diagram illustrates how to handle the black-and-white inversion of a code using a single neural network. [Figure 23] This graph shows the relationship between contrast and matching level. [Figure 24] This is a flowchart showing an example of the matching level calculation process. [Figure 25] This figure shows an example of a user interface displayed on the display unit. [Figure 26] This figure shows an example using an image sensor with an AI chip. [Modes for carrying out the invention]

[0037] Embodiments of the present invention will be described in detail below with reference to the drawings. The following description of preferred embodiments is essentially illustrative and is not intended to limit the present invention, its applications, or its uses.

[0038] (Stationary optical information reading device) Figure 1 is a schematic diagram showing the operation of a stationary optical information reading device 1 according to an embodiment of the present invention. In this example, multiple workpieces W are placed on the upper surface of a conveyor belt B and transported in the direction of arrow Y in Figure 1, and the optical information reading device 1 according to the embodiment is installed at a distance above the workpieces W. The optical information reading device 1 is a code reader configured to photograph the code attached to the workpieces W and decode the code contained in the captured image to read the information. In the example shown in Figure 1, the optical information reading device 1 is stationary. When operating this stationary optical information reading device 1, it is fixed to a bracket or the like (not shown) to prevent it from moving. Alternatively, the stationary optical information reading device 1 may be used while being held by a robot (not shown). Furthermore, the optical information reading device 1 may be used to read the code of a stationary workpiece W. The operation of the stationary optical information reading device 1 refers to the time when it is sequentially reading the codes of the workpieces W being transported by the transport belt conveyor B.

[0039] Furthermore, each workpiece W has a code attached to its outer surface. The code includes both barcodes and two-dimensional codes. Examples of two-dimensional codes include QR code (registered trademark), micro QR code, Data matrix (Data code) (registered trademark), Veri code, Aztec code, PDF417, and Maxi code. Two-dimensional codes come in stacked and matrix types, but the present invention is applicable to any type of two-dimensional code. The code may be attached to the workpiece W by printing or engraving it directly, or by printing it on a label and then attaching it to the workpiece W; the means and method are not limited.

[0040] The optical information reading device 1 is wired to the computer 100 and the programmable logic controller (PLC) 101 by signal lines 100a and 101a, respectively. However, it is not limited to this configuration; communication modules may be built into the optical information reading device 1, the computer 100, and the PLC 101 to wirelessly connect the optical information reading device 1 to the computer 100 and the PLC 101. The PLC 101 is a control device for sequence control of the conveyor belt B and the optical information reading device 1, and a general-purpose PLC can be used. The computer 100 can be a general-purpose or dedicated electronic computer or a portable terminal.

[0041] Furthermore, during operation, the optical information reading device 1 receives a reading start trigger signal from the PLC 101 via signal line 101a, which defines the start timing for code reading. Based on this reading start trigger signal, the optical information reading device 1 performs image capture and decoding of the code. Subsequently, the decoded result is transmitted to the PLC 101 via signal line 101a. In this way, during operation of the optical information reading device 1, the input of the reading start trigger signal and the output of the decoded result are repeatedly performed between the optical information reading device 1 and an external control device such as the PLC 101 via signal line 101a. Note that the input of the reading start trigger signal and the output of the decoded result may be performed via signal line 101a between the optical information reading device 1 and the PLC 101, as described above, or via other signal lines not shown. For example, a sensor for detecting the arrival of workpiece W may be directly connected to the optical information reading device 1, and the reading start trigger signal may be input from that sensor to the optical information reading device 1.

[0042] As shown in Figure 2, the optical information reading device 1 includes a box-shaped housing 2, a polarizing filter attachment 3, an illumination unit 4, a camera 5, a display unit 6, a power connector 7, and a signal line connector 8. Furthermore, the housing 2 is provided with an indicator 9, an aimer light illumination unit (light illumination unit) 10, and operation buttons 11 and 12, and the indicator 9, aimer light illumination unit 10, and operation buttons 11 and 12 are also components of the optical information reading device 1.

[0043] The housing 2 has an elongated shape in a predetermined direction, but the shape of the housing 2 is not limited to the shape shown in the illustration. A polarizing filter attachment 3 is detachably attached to the front outer surface of the housing 2. The illumination unit 4, camera 5, aimer light irradiation unit 10, processor 20, storage unit 30, ROM 40, RAM 41, etc. are housed inside the housing 2. The processor 20, storage unit 30, ROM 40, and RAM 41 are also components of the optical information reading device 1.

[0044] A lighting unit 4 is provided on the front side of the housing 2. The lighting unit 4 is a part that illuminates at least the code of the workpiece W by irradiating light toward the front of the optical information reading device 1. As shown in Figure 3, the lighting unit 4 comprises a first lighting unit 4a consisting of a plurality of light-emitting diodes (LEDs), a second lighting unit 4b consisting of a plurality of light-emitting diodes, and a lighting drive unit 4c consisting of an LED driver or the like that drives the first lighting unit 4a and the second lighting unit 4b. The first lighting unit 4a and the second lighting unit 4b are driven individually by the lighting drive unit 4c and can be turned on and off separately. The lighting drive unit 4c is connected to a processor 20, and the lighting drive unit 4c is controlled by the processor 20. Note that one of the first lighting unit 4a and the second lighting unit 4b may be omitted.

[0045] As shown in Figure 2, a camera 5 is provided in the central front part of the housing 2. The optical axis direction of the camera 5 is approximately the same as the direction of light irradiation by the illumination unit 4. The camera 5 is the part that photographs the code and generates a read image. As shown in Figure 3, the camera 5 includes an image sensor 5a that receives reflected light from the code attached to the workpiece W and illuminated by the illumination unit 4, an optical system 5b having a lens and the like, and an AF module (autofocus module) 5c. Light reflected from the part of the workpiece W to which the code is attached is incident on the optical system 5b, and the incident light is emitted toward the image sensor 5a and an image is formed on the imaging surface of the image sensor 5a.

[0046] The image sensor 5a is an image sensor consisting of a light-receiving element such as a CCD (charge-coupled device) or CMOS (complementary metal oxide semiconductor) that converts the image of the code obtained through the optical system 5b into an electrical signal. The image sensor 5a is connected to the processor 20, and the electrical signal converted by the image sensor 5a is input to the processor 20 as data for the read image. The AF module 5c is a mechanism that adjusts focus by changing the position and refractive index of the focusing lens among the lenses that make up the optical system 5b. The AF module 5c is also connected to the processor 20 and controlled by the processor 20.

[0047] As shown in Figure 2, a display unit 6 is provided on the side of the housing 2. The display unit 6 consists of, for example, an organic EL display or a liquid crystal display. The display unit 6 is connected to the processor 20 and can display, for example, the code captured by the imaging unit 5, the string resulting from the decoding of the code, the reading success rate, the matching level, etc. The reading success rate is the average reading success rate when the reading process is performed multiple times. The matching level is the read margin, which indicates how easy it is to read a code that has been successfully decoded. This can be determined from the number of error corrections that occurred during decoding, etc., and can be expressed as a numerical value, for example. The fewer the error corrections, the higher the matching level (read margin), and conversely, the more error corrections, the lower the matching level (read margin).

[0048] A power cable (not shown) for supplying external power to the optical information reading device 1 is connected to the power connector 7. Signal lines 100a, 101a, etc., for communication with the computer 100 and PLC 101 are connected to the signal line connector 8. The signal line connector 8 can be configured as, for example, an Ethernet connector, a serial communication connector such as RS232C, or a USB connector.

[0049] The housing 2 is provided with an indicator 9. The indicator 9 is connected to the processor 20 and can be made up of a light-emitting element such as a light-emitting diode. The operating status of the optical information reading device 1 can be communicated externally by the illumination status of the indicator 9.

[0050] A pair of aimer light emitting units 10 are provided on the front side of the housing 2, flanking the camera 5. As shown in Figure 3, the aimer light emitting unit 10 comprises an aimer 10a made of a light-emitting diode or the like, and an aimer drive unit 10b that drives the aimer 10a. The aimer 10a is used to indicate the shooting range, field of view center, and optical axis of the illumination unit 4 of the camera 5 by emitting light (aimer light) toward the front of the optical information reading device 1. Specifically, the aimer 10a irradiates visible light of a different color (e.g., red or green) than the ambient light toward the shooting field of view of the camera 5, forming a mark visible to the naked eye on the surface irradiated with the visible light. The mark may be various shapes, symbols, letters, etc. The user can also set up the optical information reading device 1 by referring to the light emitted from the aimer 10a.

[0051] As shown in Figure 2, operation buttons 11 and 12 are provided on the side of the housing 2 for use when setting the optical information reading device 1, etc. Operation buttons 11 and 12 include, for example, select buttons and enter buttons. In addition to operation buttons 11 and 12, a touch panel type operation means may also be provided. Operation buttons 11 and 12 are connected to a processor 20, and the processor 20 is capable of detecting the operation status of operation buttons 11 and 12. By operating operation buttons 11 and 12, it is possible to select one of several options displayed on the display unit 6 and to confirm the selected result.

[0052] (Handheld optical information reading device) The above example shows a case where the optical information reading device 1 is stationary, but the present invention is applicable to optical information reading devices other than stationary ones. Figure 4 shows a handheld optical information reading device 1A, and the present invention can also be applied to a handheld optical information reading device 1A like the one shown in this figure.

[0053] The housing 2A of the handheld optical information reading device 1A is elongated in the vertical direction. While the orientation of the optical information reading device 1A is not limited to the illustrated orientation and can be used in various directions, for the sake of explanation, the vertical direction of the optical information reading device 1A is specified.

[0054] A display unit 6A is provided in the upper part of the housing 2A. The display unit 6A is configured similarly to the display unit 6 of the stationary optical information reading device 1. The lower part of the housing 2A is a gripping part 2B for the user to hold during operation. The gripping part 2B is a part that can be held in the same way that an average adult would hold it with one hand, and its shape and size can be freely set. By holding this gripping part 2B, the optical information reading device 1A can be carried and moved around. In other words, this optical information reading device 1A is a portable terminal device, and can also be called, for example, a handheld terminal.

[0055] Similar to the stationary optical information reading device 1, the handheld housing 2A also houses an illumination unit, camera, aimer light irradiation unit, processor, memory unit, ROM, RAM, etc. (not shown). The optical axes of the illumination unit, the camera, and the aimer light irradiation unit are directed diagonally upward from near the top end of the housing 2A. The handheld housing 2A is also equipped with a buzzer (not shown).

[0056] Multiple operation buttons 11A and a trigger key 11B are provided on or near the gripping section 2B. The operation buttons 11A are the same as the operation buttons 11 of the stationary optical information reading device 1. When the user points the tip (upper end) of the optical reading device 1A towards the workpiece W and presses the trigger key 11B, an aimer light is emitted from the tip of the optical reading device 1A. The user adjusts the orientation of the optical reading device 1A while visually observing the aimer light reflected from the surface of the workpiece W, and when the aimer light is aligned with the code to be read, the code reading and decoding process is performed automatically. When reading is complete, a completion notification sound is emitted from the buzzer.

[0057] One example of the use of the handheld optical information reader 1A is its application in picking operations within a logistics warehouse. For instance, when shipping ordered goods from a logistics warehouse, the necessary items are picked from the shelves within the warehouse. This picking process involves a user holding an order slip with a code, moving to the shelves, and then comparing the code on the order slip with the code attached to the product or shelf. In this case, the handheld optical information reader 1A is used to alternately read the code on the order slip and the code attached to the product or shelf.

[0058] (Processor configuration) The following description applies to both the stationary optical information reading device 1 and the handheld optical information reading device 1A, and can be applied to either device 1 or 1A unless otherwise specified. As shown in Figure 3, the processor 20 comprises a CPU core 21, a DSP core 22, and an AI chip 23. The CPU core 21, DSP core 22, and AI chip 23 are a so-called System-on-a-chip (SoC, SOC) and are mounted on a single substrate. Note that the DSP core 22 and AI chip 23 do not have to be an SoC, in which case they would not be mounted on the same substrate, but this case is also included within the scope of the present invention.

[0059] High-speed RAM 41 is connected to the CPU core 21, DSP core 22, and AI chip 23, and all three can access RAM 41. In addition, ROM 40 is connected to the processor 20, and all three can access ROM 40.

[0060] The CPU core 21 is a general-purpose processor that performs tasks such as AF control, lighting control, camera control, and decoding of scanned images. The DSP core 22 performs tasks such as various filtering processes on scanned images. The AI ​​chip 23 is an integrated circuit dedicated to the trial of image restoration using a neural network, and is specialized for performing multiply-accumulate operations necessary for neural network processing at extremely high speed.

[0061] As shown in Figure 5, the processor 20 comprises an AF control unit 20a, an imaging control unit 20b, a filter processing unit 20c, a tuning execution unit 20d, a decoding processing unit 20f, an extraction unit 20g, a reduction unit 20h, an enlargement unit 20i, and an image repair unit 20j. The AF control unit 20a, imaging control unit 20b, filter processing unit 20c, tuning execution unit 20d, decoding processing unit 20f, an extraction unit 20g, a reduction unit 20h, and an enlargement unit 20i are parts that are processed by the calculations of the CPU core 21 or the DSP core 22. On the other hand, the image repair unit 20j is a part that is processed by the AI ​​chip 23.

[0062] (Configuration of the AF control unit) The AF control unit 20a is a unit that controls the AF module 5c shown in Figure 3, and is configured to focus the optical system 5b using conventionally known contrast AF or phase-detection AF.

[0063] (Configuration of the imaging control unit) The imaging control unit 20b is a unit that adjusts the gain of the camera 5, controls the light intensity of the illumination unit 4, and controls the exposure time (shutter speed) of the image sensor 5a. Here, the gain of the camera 5 refers to the amplification ratio (also called magnification) when the brightness of the image output from the image sensor 5a is amplified by digital image processing. The light intensity of the illumination unit 4 can be changed by separately controlling the first illumination unit 4a and the second illumination unit 4b. The gain, the light intensity of the illumination unit 4, and the exposure time are imaging conditions for the camera 5.

[0064] (Configuration of the filter processing unit) The filter processing unit 20c is the part that performs image processing filters on the read image. The filter processing unit 20c performs noise reduction filters to remove noise contained in the image generated by the camera 5, contrast correction filters to correct contrast, averaging filters, etc. The image processing filters performed by the filter processing unit 20c are not limited to noise reduction filters, contrast correction filters, and averaging filters, but may also include other image processing filters.

[0065] The filter processing unit 20c is configured to execute an image processing filter on the read image before image restoration, as described later. The filter processing unit 20c is also configured to execute an image processing filter on the read image before enlargement or reduction, as described later. Furthermore, the filter processing unit 20c may be configured to execute an image processing filter on the read image after image restoration, or on the read image after enlargement or reduction.

[0066] (Configuration of the tuning execution unit) The tuning execution unit 20d shown in Figure 5 repeatedly performs imaging and decoding by changing the imaging conditions of the camera 5 and the decoding conditions of the decoding process. Based on the matching level, which indicates the ease of reading the code calculated for each imaging and decoding condition, it determines the optimal imaging and decoding conditions and performs tuning to set the size of the code to be read. Specifically, the tuning execution unit 20d is the part that sets various conditions (tuning parameters) to make the conditions suitable for decoding by changing the imaging conditions such as the gain of the camera 5, the light intensity and exposure time of the illumination unit 4, and the image processing conditions in the filter processing unit 20c when setting up the optical information reading devices 1 and 1A. The image processing conditions in the filter processing unit 20c include the coefficient of the image processing filter (strength of the filter), switching between image processing filters if there are multiple image processing filters, and combinations of different types of image processing filters. Appropriate imaging and image processing conditions differ depending on the influence of ambient light on the workpiece W during transport, and the color and material of the surface to which the code is attached. Therefore, the tuning execution unit 20d searches for more appropriate imaging conditions and image processing conditions and sets the processing performed by the AF control unit 20a, imaging control unit 20b, and filter processing unit 20c.

[0067] The tuning execution unit 20d is configured to obtain the size of the codes contained in the image read by the camera 5. When obtaining the size of a code, the tuning execution unit 20d first searches for a code based on features that indicate code-likeness. Next, the tuning execution unit 20d obtains the PPC, code type, code size, etc., of the searched code. The PPC, code type, code size, etc., are included in the code parameters or code conditions, and therefore the tuning execution unit 20d is configured to obtain the code parameters or code conditions of the searched code.

[0068] A two-dimensional code is composed of multiple white and black modules arranged randomly. For example, in QR code Model 2, the number of modules ranges from 21x21 to 177x177. In the case of a stationary optical information reader 1, the number of modules and PPC are limited by reading the code to be read in advance, and the size of the code (pixel size) is limited to the number of modules × PPC.

[0069] PPC is a code parameter that indicates how many pixels (picture elements) make up a particular module among the modules that make up the code. The tuning execution unit 20d can obtain the PPC by identifying a module and then counting the number of pixels that make up that module.

[0070] The code type refers to the type of code, such as QR code, data matrix code, or Vericode. Each code has its own characteristics, so the tuning execution unit 20d can determine the code type, for example, by the presence or absence of a finder pattern.

[0071] Furthermore, the code size can be calculated from the number of modules arranged vertically or horizontally in the code and the PPC. The tuning execution unit 20d calculates the number of modules arranged vertically or horizontally in the code, calculates the PPC, and obtains the code size by multiplying the number of modules arranged vertically or horizontally by the PPC.

[0072] (Configuration of the decoding processing unit) The decoding unit 20f is responsible for decoding the black and white binarized data. A table showing the correspondence between encoded data can be used for decoding. Furthermore, the decoding unit 20f checks whether the decoded result is correct according to a predetermined checking method. If an error is found in the data, an error correction function is used to calculate the correct data. The error correction function varies depending on the type of code.

[0073] In this embodiment, the decoding processing unit 20f decodes the codes contained in the read image after image restoration, as described later, but it can also decode the codes contained in the read image before image restoration. The decoding processing unit 20f is configured to write the decoding result obtained by decoding the codes to the decoding result storage unit 30b of the storage unit 30 shown in Figure 3.

[0074] (Configuration of the extraction unit) The extraction unit 20g is responsible for extracting code candidate regions from the read image generated by the camera 5 that have a high probability of containing a code. Code candidate regions can be extracted based on features that indicate code-likeness, in which case the features that indicate code-likeness serve as information for identifying the code. For example, the extraction unit 20g can acquire a read image and search for a code in the acquired read image based on features that indicate code-likeness. Specifically, it searches whether there are any parts in the acquired read image that have a predetermined or higher level of features that indicate code-likeness, and if it can find a part with features that indicate code-likeness, it extracts the region containing that part as a code candidate region. A code candidate region may contain areas other than codes, but it will at least contain areas that have a predetermined or higher probability of being codes. It should be noted that a code candidate region is only a region that has a high probability of containing a code, so it is possible that it may not contain a code.

[0075] The extraction unit 20g can also use the positional information of the aimer light emitted from the aimer light irradiation unit 10 as information for identifying the code and extract a code candidate area. In this case, the extraction unit 20g identifies a portion of the read image that corresponds to the marker formed by the aimer light irradiation unit 10 and extracts the identified portion as a code candidate area. That is, as described above, since the aimer light is used to indicate the shooting range and field of view center of the camera 5, the marker formed by the aimer light can be pre-aligned with the field of view center of the camera 5. In particular, in the case of the handheld optical information reading device 1A, the user aligns the aimer light with the code to be read, so there is a high probability that the marker formed by the aimer light will overlap with the code. In other words, by aligning the portion of the marker formed by the aimer light irradiation unit 10 with the field of view center of the camera 5, when the extraction unit 20g extracts an area that includes the field of view center of the camera 5, that area is a region that is highly likely to contain a code. In this case, the portion corresponding to the marker formed by the aimer light irradiation unit 10 does not need to perfectly coincide with the center of the camera 5's field of view, and may be slightly offset in the vertical or horizontal direction of the field of view. Also, since the code has a predetermined size, the area extracted by the extraction unit 20g is not only the center of the camera 5's field of view, but also an area of ​​a predetermined size that includes the center of the field of view.

[0076] The extraction unit 20g can also identify the central portion of the read image and extract that central portion as a candidate code area. For example, by pre-acquiring the center of the camera 5's field of view as information for identifying the code, the portion corresponding to the center of the camera 5's field of view can be identified on the read image. In particular, in the case of the handheld optical information reading device 1A, the central portion identified on the read image corresponds to the marker formed by the aimer light irradiation unit 10, so that central portion is an area where a code is highly likely to exist.

[0077] The extraction unit 20g can be configured to accept a user's selection of a predetermined portion from the read image, and can extract the selected portion as a code candidate region. That is, when a user specifies a region of any size at any position (coordinate) on the read image, the extraction unit 20g accepts the selection of a predetermined portion from the read image based on the coordinate and size information of that position. The extraction unit 20g extracts the accepted portion as a code candidate region. For example, in the case of a stationary optical information reading device 1, as shown in Figure 1, the device photographs a workpiece W being transported by a transport belt conveyor B and performs code decoding. However, the workpiece W is not necessarily located in the center of the transport belt conveyor B in the width direction; it may be located at the edge. In this case, by specifying the portion corresponding to the edge of the transport belt conveyor B from the read image, it becomes possible to accurately extract a region that is highly likely to contain a code. In addition, in the case of a large workpiece W, the code may be displayed near the edge, away from the center. In this case, by specifying the area near the edge of the workpiece W from the read image, it becomes possible to accurately extract the region where the code is most likely to exist.

[0078] (Memory Unit Configuration) The memory unit 30 shown in Figure 3 can be configured as a read / write storage device such as an SSD (Solid State Drive), but each of the memory units 30a, 30b, 30c, and 30d described below may be provided in the ROM 40 instead of the above storage device. That is, in this embodiment, a configuration in which the image data storage unit 30a, the decode result storage unit 30b, the parameter set storage unit 30c, and the neural network storage unit 30d are included in the ROM 40 is also applicable. The image data storage unit 30a is the part that stores the images read by the camera 5. The decode result storage unit 30b is the part that stores the decoding results of the code executed by the decode processing unit 20f. The parameter set storage unit 30c is the part that stores the various conditions set as a result of tuning performed by the tuning execution unit 20d and various conditions set by the user. The neural network storage unit 30d is the part that stores the structure and parameters of the trained neural network, which will be described later.

[0079] (Image restoration using neural networks) The optical information reading devices 1 and 1A have an image restoration function that restores the images read by the camera 5 using a neural network. This embodiment differs from conventional optical information reading devices in that, instead of performing machine learning in the optical information reading devices 1 and 1A, the optical information reading devices 1 and 1A have pre-stored the structure and parameters of a neural network that has already been trained, and the image restoration is performed using a neural network composed of the structure and parameters that have been stored.

[0080] As shown in Figure 6, a neural network has an input layer into which input data (image data in this example) is input, an output layer that outputs output data, and an intermediate layer placed between the input and output layers. Multiple intermediate layers can be provided, for example, to create a multi-layered neural network.

[0081] (Neural network training) First, the basic procedure for training a neural network will be explained based on the flowchart shown in Figure 7. Neural network training can be performed using a computer prepared for training purposes other than the optical information reading devices 1 and 1A, but it may also be performed using a general-purpose computer not specifically for training. Furthermore, conventionally known methods may be used for training the neural network.

[0082] In step SA1 after the start, pre-prepared data of defective images is read. Defective images are images that have parts unsuitable for reading the code, and examples include the image shown on the left side of Figure 8. Unsuitable parts include, for example, dirty parts or colored parts. Then, the process proceeds to step SA2, where pre-prepared data of ideal images is read. Ideal images are images that are suitable for reading the code, and examples include the image shown on the right side of Figure 8. Defective images and ideal images are kept as pairs, and multiple such pairs are prepared.

[0083] In step SA2, the loss function is calculated to find the difference between the defective image and the ideal image. In step SA3, the neural network parameters are updated to reflect the difference found in step SA2.

[0084] Step SA4 determines whether the machine learning completion conditions are met. The machine learning completion conditions can be set based on the difference obtained in Step SA2. For example, if the difference is less than or equal to a predetermined value, it can be determined that the machine learning completion conditions are met. If Step SA4 determines YES and the machine learning completion conditions are met, the machine learning process ends. On the other hand, if Step SA4 determines NO and the machine learning completion conditions are not met, the process returns to Step SA1, loads another defective image, and then proceeds to Step SA2 to load another ideal image that is paired with the defective image in question.

[0085] In this way, a pre-trained neural network can be generated by machine learning with multiple defective images and multiple ideal images corresponding to each of those defective images. By having the optical information reading device 1, 1A store the structure and parameters of the pre-trained neural network, the pre-trained neural network can be constructed within the optical information reading device 1, 1A. That is, when performing machine learning, a huge number of defective images and ideal images are input into the neural network, so extremely high computational power is required at that time. Securing such high computational power within the optical information reading device 1, 1A is difficult, as it may not be possible to meet the requirements for miniaturization and weight reduction of the device. As in this example, if machine learning is performed outside the optical information reading device 1, 1A, and the structure and parameters of the resulting neural network are stored in the optical information reading device 1, 1A, it becomes possible to perform image restoration using a neural network while achieving miniaturization and weight reduction of the optical information reading device 1, 1A.

[0086] Figure 9 is a conceptual diagram of a convolutional neural network (CNN) constructed as described above. In "Convolution," features of the input image are extracted. This layer consists of convolutional operations, similar to image processing filters. The weights of the filter are called kernels, and feature extraction is performed according to the kernels. A convolutional layer generally has multiple different kernels, and the number of maps (D) increases according to the number of kernels. The kernel size can be, for example, 3x3 or 5x5.

[0087] In "pooling," the responses of each kernel are combined through a reduction process. In "Deconvolution," the image is reconstructed from the feature values ​​using an inverse convolution filter. In "Unpooling," the response values ​​are sharpened through an expansion process.

[0088] The structure and parameters of the trained neural network are stored in the neural network storage unit 30d of the storage unit 30 shown in Figure 3. The structure of the neural network refers to the number of hidden layers (D) between the input layer and the output layer, the number of image processing filters, etc. The parameters of the neural network are the parameters set in step SA3 of the flowchart shown in Figure 7.

[0089] Furthermore, the neural network memory unit 30d is capable of storing the structures and parameters of multiple neural networks with different numbers of layers or image processing filters. In this case, the structures and parameters of multiple neural networks can be stored in the neural network memory unit 30d in association with coding conditions. For example, the structure and parameters of a neural network for constituting a first neural network can be associated with a first coding condition corresponding to the first neural network. When making this association, the number of layers or filters of the first neural network can be determined in advance by the first coding condition, particularly the first PPC, so the association is made in such a way that this relationship is maintained. Similarly, the structure and parameters of a second neural network, which is different from the first neural network, can be associated with a second coding condition, which is different from the first coding condition.

[0090] When the structures and parameters of multiple neural networks are stored in the neural network storage unit 30d, the tuning execution unit 20d operates as follows when setting up the optical information reading devices 1 and 1A. Specifically, when setting the code conditions included in the read image generated by the camera 5, the tuning execution unit 20d reads the structure and parameters of the neural network associated with the set code conditions from the neural network storage unit 30d and identifies it as the neural network to be used during operation. For example, if the tuning execution unit 20d sets a first code condition as the code condition included in the read image, it reads the structure and parameters of the first neural network associated with the first code condition from the neural network storage unit 30d. Then, it identifies the structure and parameters of the first neural network as the neural network to be used during the operation of the optical information reading devices 1 and 1A. This allows the system to attempt to repair the read image with the neural network structure and parameters that are optimal for the code conditions, thereby improving the reading accuracy.

[0091] When training a neural network, it is also possible to train it using defective and ideal images where the pixel resolution of the modules that make up the code is within a specific range. The pixel resolution of a module can be represented, for example, by the number of pixels (PPC) of one module that makes up the code, and machine learning of the neural network can be performed only on defective and ideal images where the PPC is within a specific range.

[0092] The specified range described above includes pixel resolutions that offer a high image restoration effect, while excluding pixel resolutions that show little improvement in image restoration. This range can be set assuming the restoration of images containing code composed of multiple modules. This allows for improved processing speed by creating a neural network structure specifically designed for restoring images containing code.

[0093] The specified range for the number of pixels mentioned above can be, for example, 4 PPC or more and 6 PPC or less, but it may also be less than 4 PPC or more than 6 PPC. Furthermore, by limiting the specified range to 4 PPC or more and 6 PPC or less for machine learning, it is possible to eliminate unnecessary scale variations from the images used for training and optimize the structure of the trained neural network.

[0094] (Attempting to repair) As shown in Figure 5, the processor 20 of the optical information reading devices 1 and 1A is equipped with an image repair unit 20j, which is composed of an AI chip 23. The image repair unit 20j reads the structure and parameters of the neural network stored in the neural network storage unit 30d of the storage unit 30, and configures a neural network within the optical information reading devices 1 and 1A using the read neural network structure and parameters. The neural network configured within the optical information reading devices 1 and 1A is the same as the trained neural network that was trained using the procedure shown in the flowchart of Figure 7.

[0095] The image restoration unit 20j inputs the image generated by the camera 5 into a neural network and attempts to restore the image according to the structure and parameters of the neural network stored in the neural network memory unit 30d. Then, the decoding processing unit 20f performs a decoding process on the restored image.

[0096] The filter processing unit 20c is configured to perform image processing filters on the image read by the camera 5 before the image restoration unit 20j inputs the image to the neural network. This allows appropriate image processing to be performed before attempting restoration by the neural network, resulting in more accurate restoration of the read image.

[0097] The image repair unit 20j may attempt image repair for all read images decoded by the decoding processing unit 20f, or it may attempt image repair for only some of the read images decoded by the decoding processing unit 20f. For example, the decoding processing unit 20f performs decoding on the read image before attempting repair and determines whether the decoding was successful. If the decoding is successful, it means that the read image did not require repair, and the result is output as is. On the other hand, if decoding fails, the image repair unit 20j attempts to repair the read image, and the decoding processing unit 20f performs decoding on the repaired read image.

[0098] (Input of a partial image) The image that the image restoration unit 20j inputs to the neural network may be the entirety of the images read by the camera 5. However, the larger the size of the input image, the greater the computational load on the neural network, which may lead to a decrease in processing speed. Specifically, in a Fully Convolutional Network (FCN) type neural network as shown in Figure 9, assuming the same structure, the computational load will be proportional to the input image size, and since the input image size is proportional to the square of the code size, the computational load will also be proportional to the square of the code size.

[0099] On the other hand, it is virtually impossible for the entire image read by camera 5 to contain the code. Under normal operation, the code is only present in a portion of the read image. If only that portion can be repaired using a neural network, the reading accuracy can be improved.

[0100] Therefore, the image input to the neural network by the image repair unit 20j can be a portion of the image read by the camera 5, i.e., a partial image containing the code. Specifically, the image repair unit 20j cuts out a partial image corresponding to the code candidate region extracted by the extraction unit 20g from the image read by the camera 5. By inputting the cut-out partial image into the neural network, the image repair unit 20j attempts to repair the partial image according to the structure and parameters stored in the neural network memory unit 30d. Then, the decoding processing unit 20f performs a decoding process on the repaired partial image.

[0101] As mentioned above, since the code is rarely present throughout the entire image being read, the size of the partial image corresponding to the code candidate region will be smaller than the size of the read image. This reduces the size of the image input to the neural network, thereby lowering the computational load on processor 20 and ultimately resulting in faster processing speed. Furthermore, since the partial image corresponds to a region where the code is most likely to exist, it is possible to create an image that contains all the information necessary for reading, i.e., the entire code. By inputting this image containing the entire code into the neural network, the decrease in reading accuracy is suppressed.

[0102] The image restoration unit 20j can determine the input size of a partial image to the neural network based on the code size obtained by the tuning execution unit 20d. As described above, the tuning execution unit 20d can obtain the code size. The image restoration unit 20j increases the input size of the partial image to be input to the neural network by a predetermined amount compared to the code size obtained by the tuning execution unit 20d.

[0103] In other words, while inputting a partial image into the neural network can reduce the computational load, if, for example, the code is rotated or the position of the code cannot be precisely detected, a portion of the code in the partial image may be missing, potentially leading to a failure in the decoding process. In this embodiment, the input size to the neural network is not the same as the code size acquired by the tuning execution unit 20d, but is larger by a predetermined amount than the code size, thereby suppressing the loss of a portion of the code in the partial image. Furthermore, by making it larger by a predetermined amount than the code size, an upper limit can be placed on the input size to the neural network, thus preventing the computational load from becoming heavy. The "determined amount" may be, for example, the horizontal (or vertical) length of the partial image input to the neural network being 1.5 times or more, 2.0 times or more, or 3.0 times or more, the horizontal (or vertical) length of the code acquired by the tuning execution unit 20d.

[0104] When the image restoration unit 20j determines the input size of a partial image to be input to the neural network, it can be determined based on the number of pixels in one module that makes up the code acquired by the tuning execution unit 20d and the number of modules arranged vertically or horizontally in that code.

[0105] Furthermore, the tuning execution unit 20d can set code conditions as described above. The image repair unit 20j can also set the size of the read image to be input to the neural network based on the code conditions set by the tuning execution unit 20d. For example, if the image repair unit 20j obtains the code size from the code conditions set by the tuning execution unit 20d, it inputs a partial image that is a predetermined amount larger than the obtained code size to the neural network. If the code size is large, the size of the partial image input to the neural network will be large, while if the code size is small, the size of the partial image input to the neural network will be small. In other words, the size of the partial image input to the neural network can be changed according to the code conditions.

[0106] (Zoom in / out function) In a convolutional neural network like the one shown in Figure 9, in order to adequately capture the features of a code and restore it to an image suitable for decoding, it is necessary to extract features within a range that covers a certain number of modules. For example, if one feature value obtained from the neural network is extracted from a range of 6x6 modules, and the image is captured with one module being 20 pixels, then the neural network must be designed to calculate the aggregated feature value from a range of 120x120 pixels. In other words, the wider the pixel range to be covered, the deeper the neural network must be. For example, to aggregate feature values ​​from the aforementioned 120x120 pixel range, six convolutional layers are required.

[0107] However, since the code is composed of randomly arranged white and black modules, the relationships between pixel values ​​over a wide range are weak, and the restoration effect hardly improves even if the covered pixel range is expanded beyond a certain point. In this specification, such characteristics are referred to as narrow-range features.

[0108] Furthermore, while the arrangement of fixed module patterns, such as finder patterns, within the code, and the fact that the code as a whole is rectangular, can be considered broad-range features, if the goal is simply to roughly separate modules from the background, it is possible to sufficiently restore the code using only narrow-range features without needing broad-range features. Broad-range features can be defined as features that have a fixed shape over a wide area.

[0109] Therefore, when training the neural network shown in Figure 7, images in which the PPC of the code is within a specific range can be used. This improves processing speed by creating a neural network structure specialized for repairing images containing code, but there is a concern that the repair effect on read images where the PPC is outside the specific range will decrease. Note that when training the neural network, images in which the PPC of the code is outside the specific range may also be used.

[0110] In the optical information reading devices 1 and 1A of this embodiment, a function for reducing and enlarging the read image is provided to ensure that the PPC of the read image falls within a specific range. As shown in Figure 5, the processor 20 is configured with a reduction unit 20h and an enlargement unit 20i. The reduction unit 20h is the part that generates a read image reduced so that the pixel resolution of the modules constituting the code in the read image generated by the camera 5 falls within the specified range described above, and the enlargement unit 20i is the part that generates a read image enlarged so that the pixel resolution of the modules constituting the code in the read image generated by the camera 5 falls within the specified range described above.

[0111] The reduction unit 20h and the enlargement unit 20i determine, for example, whether the PPC of the code in the image read by the camera 5 is outside a specific range (4 PPC or more and 6 PPC or less). If it is within the specific range, reduction or enlargement is not performed; if it is outside the specific range, reduction or enlargement is performed. By performing reduction or enlargement, the PPC of the code in the partial image that the image restoration unit 20j inputs to the neural network comes within the specific range.

[0112] The image restoration unit 20j attempts to restore the read image according to its structure and parameters by inputting the read image, which has been enlarged or reduced to fit within a specific range, into a neural network configured with the structure and parameters stored in the neural network storage unit 30d. Then, the decoding processing unit 20f performs a decoding process on the restored read image.

[0113] The extraction unit 20g may be configured to search for codes in the read image enlarged by the enlargement unit 20i or reduced by the reduction unit 20h, and to extract the region containing the searched code as a code candidate region where the code is likely to exist. In this case, the image repair unit 20j attempts to repair the read image according to its structure and parameters by inputting the partial image corresponding to the code candidate region extracted by the extraction unit 20g into a neural network configured with the structure and parameters stored in the neural network storage unit 30d.

[0114] When setting up the optical information reading devices 1 and 1A, the tuning execution unit 20d can generate multiple reading images (magnified images) with different magnification ratios, and multiple reading images (reduced images) with different reduction ratios. The tuning execution unit 20d inputs the generated multiple reading images (magnified or reduced images) into a neural network to attempt to repair each reading image, performs a decoding process on each repaired reading image, and determines a read margin that indicates how easy it is to read the code. The tuning execution unit 20d identifies the magnification or reduction ratio of the reading image whose determined read margin is higher than a predetermined value as the magnification or reduction ratio to be used during operation. When the tuning execution unit 20d has identified a magnification or reduction ratio, during operation of the optical information reading devices 1 and 1A, the reduction unit 20h and the magnification unit 20i enlarge or reduce the reading image at the magnification or reduction ratio identified by the tuning execution unit 20d.

[0115] In other words, for example, if a specific range has a certain width, changing the magnification or reduction ratio even within that specific range may change the read margin. The magnification or reduction ratio of the read image during operation can be specified so that the read margin determined by the tuning execution unit 20d is higher than a predetermined value, thereby improving processing speed and reading accuracy.

[0116] The tuning execution unit 20d determines the reading margin for each read image, and then identifies the magnification or reduction ratio of the read image with the highest reading margin among the multiple read margins determined, as the magnification or reduction ratio to be used during operation. This further improves processing speed and reading accuracy.

[0117] Furthermore, by utilizing the ability to reduce and enlarge the read image using the reduction unit 20h and the enlargement unit 20i, the number of parameter sets can be increased. Multiple parameter sets with different reduction ratios by the reduction unit 20h and multiple parameter sets with different enlargement ratios by the enlargement unit 20i can be generated, and these parameter sets can be stored in the parameter set storage unit 30c of the storage unit 30. For example, if the reduction ratio by the reduction unit 20h can be set to multiple values ​​such as 1 / 2, 1 / 4, and 1 / 8, a parameter set with a reduction ratio of 1 / 2, a parameter set with a reduction ratio of 1 / 4, and a parameter set with a reduction ratio of 1 / 8 can be stored in the parameter set storage unit 30c. Similarly, if the enlargement ratio by the enlargement unit 20i can be set to multiple values ​​such as 2x, 4x, and 8x, a parameter set with an enlargement ratio of 2x, a parameter set with an enlargement ratio of 4x, and a parameter set with an enlargement ratio of 8x can be stored in the parameter set storage unit 30c. Then, any one parameter set can be applied from among the multiple parameter sets stored in the parameter set storage unit 30c.

[0118] (Image restoration filter) The image repair unit 20j may be a part that executes an image repair filter that attempts to repair the read image using a neural network. The image repair filter is a filter that attempts to repair the read image according to the structure and parameters stored in the neural network storage unit 30d by inputting the read image into a neural network composed of structures and parameters stored in the neural network storage unit 30d, and performs the same function as the image repair described above.

[0119] To enable the execution of an image repair filter, a setting unit 20e (shown in Figure 5) for setting the image repair filter can be provided. The setting unit 20e is configured to accept the setting of the image repair filter by the user. For example, when setting the optical information reading devices 1 and 1A, the setting unit 20e can be configured to generate a user interface that allows the user to select either "apply the image repair filter" or "not apply the image repair filter" and display it on the display unit 6, thereby accepting the user's selection. Therefore, the image repair unit 20j attempts to repair the read image by applying the image repair filter set by the setting unit 20e to the read image.

[0120] The setting unit 20e may be included in the tuning execution unit 20d, or it may be configured separately from the tuning execution unit 20d. If the setting unit 20e is included in the tuning execution unit 20d, it becomes possible to set parameters related to the image repair filter during tuning. Parameters related to the image repair filter may include "apply image repair filter" and "do not apply image repair filter". "Apply image repair filter" means setting the image repair filter to be executable, and "do not apply image repair filter" means not setting the image repair filter.

[0121] The parameters related to the image restoration filter are also information about the image restoration filter, and in this case, a parameter set containing information about the settings of the image restoration filter can be stored in the parameter set storage unit 30c. Since the parameters related to the image restoration filter include "application of the image restoration filter" and "non-application of the image restoration filter," the parameter set stored in the parameter set storage unit 30c includes a first parameter set that enables the image restoration filter and a second parameter set that does not set the image restoration filter. When the optical information reading devices 1 and 1A are in operation, one of the parameter sets selected from the first parameter set and the second parameter set is applied. When the first parameter set is applied, the decoding process by the decoding processing unit 20f is performed on the read image in which image restoration has been performed by the image restoration filter, while when the second parameter set is applied, the decoding process by the decoding processing unit 20f is performed on the read image in which image restoration has not been performed.

[0122] The parameter set stored in the parameter set storage unit 30c also includes items for setting the code conditions included in the image read by the camera 5. The items for setting the code conditions include PPC, code type, code size, etc., acquired by the tuning execution unit 20d. For example, one parameter set may include PPC, code type, and code size as items for setting the code conditions, the gain of the camera 5, the light intensity of the illumination unit 4, and the exposure time as imaging conditions, and the image processing filter type, parameters of the image processing filter, and application items of the image repair filter as items for the image processing filter applied by the filter processing unit 20c. The values ​​set by the tuning execution unit 20d may be used as is for each of these items, or the user may change them as they see fit.

[0123] (An example of the tuning process procedure) An example of the tuning process performed by the tuning execution unit 20d when setting up the optical information reading devices 1 and 1A will be specifically explained based on the flowchart shown in Figure 10. In step SB1 after the start of the flowchart shown in Figure 10, the tuning execution unit 20d controls the illumination unit 4 and the camera 5 to cause the camera 5 to generate a read image, and the tuning execution unit 20d acquires the read image. At this time, the presence and type of image processing filter to be executed before the decoding process, and the decoding processing parameters for applying image restoration by a neural network are set to arbitrary parameters. Next, the process proceeds to step SB2, where the tuning execution unit 20d causes the decoding processing unit 20f to perform the decoding process on the acquired read image.

[0124] After the decoding process, the process proceeds to step SB3, where the tuning execution unit 20d determines whether the decoding process in step SB2 was successful or not. If step SB3 determines NO and the decoding process in step SB2 fails, i.e., the code could not be read, the process proceeds to step SB4, where the decoding process parameters are changed to different parameters, and then the decoding process is executed again in step SB2. If the decoding process fails with all decoding process parameters, this flow is terminated and the user is notified.

[0125] On the other hand, if step SB3 is determined to be YES and the decoding process in step SB2 is successful, the process proceeds to step SB5, where the tuning execution unit 20d determines the code parameters (PPC, code type, code size, etc.). Once the tuning execution unit 20d determines the code parameters, the structure and parameters of the neural network stored in the neural network storage unit 30d in association with the code parameters are also determined (step SB6). In step SB6, the neural network is constructed using the structure and parameters read from the neural network storage unit 30d.

[0126] In step SB7, the image repair unit 20j attempts to repair the read image by inputting it into the neural network configured in step SB6, and the decoding processing unit 20f performs decoding on the repaired read image. Then, the process proceeds to step SB8, where the tuning execution unit 20d evaluates the read margin based on the decoding result of step SB7 and stores it temporarily.

[0127] Step SB9 determines whether the decoding process has been completed for all decoding parameters. If the result in Step SB9 is NO, and the decoding process has not been completed for all decoding parameters, the process proceeds to Step SB10, where the decoding parameters are changed to different parameters, and then to Step SB7.

[0128] On the other hand, if step SB9 is determined to be YES and the decoding process is completed for all decoding parameters, the process proceeds to step SB11. In step SB11, the tuning execution unit 20d selects the decoding parameter with the highest read margin from among all decoding parameters and determines that the selected decoding parameter will be applied during operation. In addition, during the tuning process, imaging conditions and other conditions are also set to appropriate conditions.

[0129] (Decoding procedure before determining reduction and enlargement ratios) Next, an example of the decoding process procedure before determining the reduction and enlargement ratios will be specifically explained based on the flowchart shown in Figure 11. The process identified in the flowchart shown in Figure 11 can be performed in step SB2 of the flowchart shown in Figure 10.

[0130] In step SC1, following the start of the flowchart shown in Figure 11, the tuning execution unit 20d selects an arbitrary image processing filter from among multiple image processing filters. This image processing filter is the filter executed by the filter processing unit 20c. Subsequently, in step SC2, the filter processing unit 20c executes the image processing filter selected in step SC1 on the read image.

[0131] Next, the process proceeds to step SC3 to create an image pyramid. The image pyramid consists of the original read image, an image of the original read image reduced to half its size, an image of the original read image reduced to a quarter of its size, an image of the original read image reduced to an eighth of its size, and so on. The reduction of the read image is performed by the reduction unit 20h. Alternatively, the image pyramid can also consist of the original read image, an image of the original read image enlarged to twice its size, an image of the original read image enlarged to four times its size, an image of the original read image enlarged to eight times its size, and so on. The enlargement of the read image is performed by the enlargement unit 20i. Note that either the reduced image or the enlarged image may be omitted.

[0132] After creating the image pyramid, the process proceeds to step SC4, where an arbitrary image is selected from among the multiple images that make up the image pyramid. This selected image may include the original image that has not been reduced or enlarged. In step SC5, the extraction unit 20g extracts code candidate regions from the image selected in step SC4 that are highly likely to contain codes. Then, the process proceeds to step SC6, where the image repair unit 20j attempts to repair the image by inputting the partial images corresponding to the code candidate regions extracted in step SC5 into the neural network. The size of the partial images input into the neural network is determined by the code conditions described above.

[0133] After repairing the read image, the process proceeds to step SC7, where the code outline and the positioning of the modules constituting the code are performed. After the positioning process, the process proceeds to step SC8, where it is determined whether each module constituting the code is white or black. After determining whether the image is white or black, the process proceeds to step SC9, where the decoding processing unit 20f performs the decoding process on the repaired read image. In the decoding process, for example, a method of reconstructing the string from the 0-1 matrix of the modules can be employed.

[0134] Next, the process proceeds to step SC10, where the tuning execution unit 20d determines whether the decoding process in step SC9 was successful or not. If step SC10 determines NO and the decoding process in step SC9 has failed, the process proceeds to step SC11, where it is determined whether all the read images constituting the image pyramid have been selected. If step SC11 determines NO and not all the read images constituting the image pyramid have been selected, the process proceeds to step SC4, where another reduced-size read image or another enlarged read image constituting the image pyramid is selected, and the process proceeds to step SC5.

[0135] On the other hand, if step SC11 is determined to be YES and all the images constituting the image pyramid are selected, this flow is terminated and the conditions for successful decoding are stored.

[0136] (Decoding procedure after determining reduction and enlargement ratios) Next, an example of the decoding process procedure after determining the reduction and enlargement ratios will be specifically explained based on the flowchart shown in Figure 12. The processes specified in the flowchart shown in Figure 12 can be performed during step SB7 of the flowchart shown in Figure 10 and during the operation of the optical information reading devices 1 and 1A.

[0137] In step SD1, following the start of the flowchart shown in Figure 12, the filter processing unit 20c executes the image processing filter selected in step SC1 of the flowchart shown in Figure 11 onto the read image. If the filter processing unit 20c executes an averaging filter on a read image like the one shown in the upper part of Figure 13, the image after the image processing filter is executed, as shown in the lower part of Figure 13, is obtained.

[0138] Next, the process proceeds to step SD2, where the read image is scaled down or enlarged as needed. Specifically, if the PPC of the code in the read image after the image processing filter has been applied is within a specific range, scaling down or enlargement is not performed. If it is outside the specific range, scaling down or enlargement is performed to bring the PPC within the specific range. An example of a read image after scaling down is shown at the top of Figure 14.

[0139] Next, in step SD3, the extraction unit 20g extracts code candidate regions that are highly likely to contain codes from the read image that was reduced or enlarged in step SD2. An example of an image with extracted code candidate regions is shown at the bottom of Figure 14. If the image was not reduced or enlarged in step SD2, the extraction process will be performed on the original read image in step SD3.

[0140] Next, the process proceeds to step SD4, where the image repair unit 20j inputs the partial image corresponding to the code candidate region extracted in step SD3 into the neural network to attempt to repair the read image. Figure 15 shows examples of the read image before and after repair. After the read image is repaired, the process ends via steps SD5 to SD7. Steps SD5 to SD7 are the same as steps SC7 to SC9 in the flowchart shown in Figure 11.

[0141] (Effects of the embodiment) As described above, according to this embodiment, when the optical information reading devices 1 and 1A are in operation, the reading image generated by the camera 5 is input to the neural network, and an attempt is made to repair the reading image. As a result, a reading image after the repair attempt is obtained, which has been repaired to be suitable for reading the code. Decoding is performed on this reading image after the repair attempt, so the reading accuracy is improved.

[0142] The neural network used for repair is constructed using a structure and parameters obtained by pre-training it with multiple defective images and multiple ideal images. Therefore, the optical information reading devices 1 and 1A only need to perform inference processing using the neural network without performing any learning processing, thereby reducing the computational load on the processor 20. As a result, the handheld optical information reading device 1A can be made smaller and lighter, and the processing speed of the stationary optical information reading device 1 can be increased.

[0143] Furthermore, when optical information reading devices 1 and 1A are in operation, if a candidate code region is extracted from the read image, the partial image corresponding to that candidate code region is input to the neural network, and an attempt is made to repair the partial image. This reduces the size of the image input to the neural network, thereby reducing the computational load on the processor 20 and consequently increasing the processing speed. In addition, since the partial image corresponds to a region where a code is likely to exist, it is possible to create an image that includes the information necessary for reading, i.e., the entire code. Because this image containing the entire code is input to the neural network, the decrease in reading accuracy is suppressed.

[0144] Furthermore, when generating the neural network structure and parameters, the system uses machine learning on defective and ideal images where the pixel resolution of the modules constituting the code is within a specific range. This allows for improved processing speed by creating a neural network structure specifically designed for repairing images containing codes. If the pixel resolution of the modules constituting the code in the read image is outside the specific range, the read image is enlarged or reduced to bring it within the specified range before being input to the neural network, enabling the repair of images containing codes with a wide range of pixel resolutions.

[0145] (Other embodiments) Figure 16 is a diagram illustrating an example of the processing procedure during operation of the optical information reading device 1 according to another embodiment. In this embodiment, the processor 20 comprises, in addition to the image restoration unit 20j, a pre-processing unit 200, a post-processing unit 201, a first decoding unit 202, and a second decoding unit 203. The pre-processing unit 200, similar to the filter processing unit 20c described above, is the part that performs noise reduction filters, contrast correction filters, averaging filters, etc. on the image read by the camera 5.

[0146] The first decoding processing unit 202 is the part that performs decoding on the image read by the camera 5, and the decoding process performed by this first decoding processing unit 202 is also called the first decoding process. If the processor 20 has multiple cores (N cores), the first decoding processing unit 202 is composed of cores 0 to i. The first decoding processing unit 202 may also be composed of a single core.

[0147] Furthermore, the second decoding processing unit 203 is the part that performs decoding on the restored image generated by the image restoration unit 20j, and the decoding process performed by this second decoding processing unit 203 is also called the second decoding process. If the processor 20 has N cores, the second decoding processing unit 203 is composed of cores i+1 to N-1. The second decoding processing unit 203 may also be composed of a single core.

[0148] As shown in Figure 17, when the first decoding processing unit 202 is composed of cores 0 to i, each of cores 0 to i extracts a code candidate region from the image read by camera 5 that is highly likely to contain a code. Then, it performs processing to determine the outline of the code and position the modules that make up the code. After that, it determines whether each module that makes up the code is white or black, and then performs the decoding process. The processing result is output to the post-processing unit 201.

[0149] On the other hand, as shown in Figure 17, when the second decoding processing unit 203 is composed of cores i+1 to N-1, each of cores i+1 to N-1 extracts code candidate regions from the read image that are highly likely to contain codes as code regions for repair, and sends a trigger signal to the image repair unit 20j to perform inference processing. In Figure 17, it is shown that one task is performed by two cores, but any one or more cores can perform one task.

[0150] Specifically, when cores N-2 and N-1, which constitute the second decoding processing unit 203, extract candidate regions for repair codes, they output a trigger signal (trigger 1) to the image repair unit 20j, causing the image repair unit 20j to perform inference processing. When the image repair unit 20j receives the trigger signal (trigger 1) sent from cores N-2 and N-1, it inputs the partial image corresponding to the candidate regions for repair codes extracted by cores N-2 and N-1 into a neural network and performs inference processing to generate a repaired image by repairing the said partial image (image repair 1). The repaired image generated by image repair 1 is sent to cores N-2 and N-1. Cores N-2 and N-1 determine grid positions indicating the positions of each cell in the code based on the repaired image generated by image repair 1, and perform decoding processing of the repaired image based on the determined grid positions.

[0151] Meanwhile, when cores i+1 and i+2, which constitute the second decoding processing unit 203, extract candidate regions for repair codes, they output a trigger signal (trigger 2) to the image repair unit 20j, causing the image repair unit 20j to perform inference processing. When the image repair unit 20j receives the trigger signal (trigger 2) sent from cores i+1 and i+2, it inputs the partial image corresponding to the candidate regions for repair codes extracted by cores i+1 and i+2 into the neural network and performs inference processing to generate a repaired image by repairing the said partial image (image repair 2). The repaired image generated by image repair 2 is sent to cores i+1 and i+2. Cores i+1 and i+2 determine grid positions indicating the positions of each cell in the code based on the repaired image generated by image repair 2, and perform decoding processing of the repaired image based on the determined grid positions.

[0152] Similarly, image repair 3 is performed by trigger signals (trigger 3) sent from core N-2 and core N-1, image repair 4 is performed by trigger signals (trigger 4) sent from core i+1 and core i+2, image repair 5 is performed by trigger signals (trigger 5) sent from core N-2 and core N-1, image repair 6 is performed by trigger signals (trigger 6) sent from core i+1 and core i+2, and image repair 7 is performed by trigger signals (trigger 7) sent from core N-2 and core N-1.

[0153] As shown in this time chart, the inference processing by the image repair unit 20j and the first decoding processing by the first decoding processing unit 202 are executed in parallel. For example, while the first decoding processing unit 202 continuously performs code region extraction, grid positioning, and decoding processing, the inference processing by the image repair unit 20j is executed once or multiple times. Also, while the first decoding processing unit 202 continuously performs code region extraction, grid positioning, and decoding processing, the second decoding processing unit 203 performs extraction of code regions for repair, output of trigger signals, grid positioning, and second decoding processing. In this way, the image repair unit 20j only needs to specialize in repairing the partial image corresponding to the candidate code regions for repair, so the inference processing speed can be increased.

[0154] During the period in which the second decoding processing unit is performing the second decoding process, there is a pause period in which the inference processing by the image repair unit 20j is not performed. That is, as shown in Figure 17, after the image repair unit 20j completes image repair 1, the second decoding processing unit 203 starts the second decoding process, but a predetermined period is provided before the image repair unit 20j starts the next image repair 2. This predetermined period is a pause period in which the image repair unit 20j does not perform image repair. This pause period is a period in which the image repair unit 20j does not perform inference processing, and by providing a pause period, the load on the image repair unit 20j can be reduced and the heat generation of the AI ​​chip 23 can be suppressed. Note that the processing time for grid positioning may be long or short depending on the image being repaired. Figure 17 shows an example in which the processing of each core is performed alternately, but depending on the processing time for grid positioning, the processing of each core is not necessarily performed alternately. That is, triggers may be issued and grid positioning may be performed in order from the core that is free to process.

[0155] As described above, the setting unit 20e is configured to allow the user to select either "apply the image repair filter" or "not apply the image repair filter." Applying the image repair filter means performing image repair, and not applying the image repair filter means not performing image repair. Therefore, the setting unit 20e also corresponds to the part that sets whether or not the second decoding processing unit 203 executes the second decoding process. Since this setting is performed by the user, the setting unit 20e is configured to accept user operations. The setting is not limited to applying or not applying the image repair filter; for example, it could also be a setting to apply or not apply inference processing (repair processing), or to apply or not apply AI processing. User operations can include, for example, operations on buttons displayed on the user interface screen.

[0156] The optical information reading device 1 is operated after a tuning process is performed to pre-determine imaging conditions such as exposure time and gain, as well as settings for image processing filters. For example, if the workpiece W is a label, the code is clearly printed, so if appropriate conditions are set through the tuning process, a high reading rate can be achieved. However, with workpieces W made of metal or resin, even if optimal conditions are set during the tuning process, reading may become difficult due to changes in the reading image caused by weak ambient light, or in some cases, reading may be impossible due to extremely low contrast caused by the optical design and algorithm design.

[0157] By executing the image restoration algorithm performed by the image restoration unit 20j in this example in parallel with the case where decoding is performed without image restoration, it is possible to improve the reading performance for workpieces W such as metal and resin while maintaining conventional reading performance, and to enhance reading stability. Furthermore, even workpieces W that cannot be read with conventional optical designs and algorithms can be read by applying the image restoration algorithm.

[0158] To give a specific example, in a process of reading a code printed on a sticker attached to a box at a certain factory, if there are no restrictions on the installation conditions of the optical information reading device 1 and optimal installation is possible, stable operation is possible even with conventional reading performance, and therefore, the image restoration filter can be omitted to prioritize cycle time.

[0159] Furthermore, in one factory, there is a process of reading codes clearly printed on metal. Although the optimal conditions are determined during the tuning process when setting up the reading system, in actual operation, reading can become unstable due to scratches on the metal surface, variations in reading position and angle, and changes in ambient brightness. In such cases, applying an image restoration filter to improve image quality makes the system more resistant to the variations in conditions mentioned above, thereby improving reading performance.

[0160] Furthermore, while some workpieces cannot be read using conventional algorithms, applying an image restoration filter can sometimes make reading possible. For example, even if the code is severely damaged or the contrast is unclear, applying an image restoration filter can repair the contrast and imperfections, making reading possible.

[0161] The second decoding process, which is performed when an image restoration filter is applied, takes longer than the first decoding process, which performs decoding on a normal read image, because it performs decoding on the restored image. Therefore, the first decoding processing unit 202 can perform the first decoding process in less time than the second decoding process performed by the second decoding processing unit 203. In this example, based on this premise, the setting unit 20e can change the decoding timeout time, which is the time limit for the decoding process.

[0162] Specifically, the setting unit 20e is configured to allow setting a decode timeout period longer than the time required to execute the second decode process when the setting unit is configured to execute the second decode process, and to allow setting a decode timeout period shorter than the time required to execute the second decode process when the setting unit is configured not to execute the second decode process. As a result, during operation, when the second decode process is executed, a decode timeout period longer than the time generally required to execute the second decode process can be set, ensuring that the decoding process for the restored image is reliably executed. On the other hand, when the setting unit is configured not to execute the second decode process, only the first decode process, which can generally be decoded in a shorter time than the second decode process, will be executed, so a short decode timeout period corresponding to the first decode process can be set. By changing the decode timeout period in this way, it is possible to set decode timeout periods corresponding to the first decode process and the second decode process, respectively.

[0163] Furthermore, although Figure 17 shows the first decoding processing unit 202 and the second decoding processing unit 203 as separate components, as shown in Figure 18, the first decoding processing unit 202 and the second decoding processing unit 203 may be combined into a single decoding processing unit 210. The decoding processing unit 210 performs the second decoding processing on the restored image in parallel with the first decoding processing on the image read by the camera 5. Thus, the parts that perform the decoding processing can be arbitrarily separated or integrated on the hardware. The following explanation will use an example where the parts that perform the decoding processing are integrated into one, but the same can be applied to cases where they are separated into multiple parts.

[0164] The decoding processing unit 210 is configured to speed up the first decoding process by allocating processing resources that would normally be used for the second decoding process to the first decoding process when the setting unit 20e has set it not to execute the second decoding process, compared to when the setting unit 20e has set it to execute the second decoding process. Processing resources refer to, for example, the memory usage area and the cores that make up the multi-core CPU. Specifically, when the setting unit 210 has set it not to execute the second decoding process, it increases the number of cores that handle the first decoding process compared to when the setting unit 20e has set it to execute the second decoding process. Also, when the setting unit 210 has set it not to execute the second decoding process, it expands the memory area used for the first decoding process compared to when the setting unit 20e has set it to execute the second decoding process. Increasing the number of cores and expanding the memory area can be done in combination, or only one of them can be done. This makes it possible to maximize the performance of the multi-core CPU and speed up the first decoding process.

[0165] Prior to image restoration, the decoding processing unit 210 extracts code candidate regions from the read image that are highly likely to contain a code. In other words, when performing image restoration using a neural network, there is a problem that the computation time becomes long if the image input to the neural network is large. Also, from the perspective of speeding up processing, it is desirable to minimize the number of decoding attempts. To address these issues, in this example, a heatmap-based search method is adopted instead of a line search-based method in order to reliably extract code candidate regions, even if it takes some time. For example, the decoding processing unit 210 quantifies the feature quantities of the code, generates a heatmap by assigning the magnitude of the feature quantities to each pixel value, and extracts code candidate regions that are highly likely to contain a code on this heatmap. A specific example is to acquire the feature portion of a 2D code (e.g., finder pattern, etc.) in the region that is relatively hot (features are large) on the heatmap. If multiple feature portions are acquired, they can be prioritized and extracted and stored in the RAM 41 or storage unit 30, etc.

[0166] After the decoding processing unit 210 extracts the code candidate region, it outputs the partial image corresponding to the extracted code candidate region to the image repair unit 20j. The image repair unit 20j inputs the partial image corresponding to the code candidate region extracted by the decoding processing unit 210 into a neural network and performs inference processing to generate a repaired image by repairing the partial image. This reduces the computation time.

[0167] Furthermore, the decoding processing unit 210 may be configured to extract code candidate regions within the read image that are large enough to contain the code to be read, as set by the tuning process described above, and that have a high probability of containing the code. The extracted image may then be reduced or enlarged to a predetermined size before being input to the neural network, and the neural network may decode the repaired image. By enlarging or reducing the extracted image to a predetermined size before inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range, and the image repair effect of the neural network can be stably obtained. In addition, although the size of the code to be read may vary, the size of the code ultimately input to the neural network can be fixed, so a balance between processing speed and ease of decoding can be maintained at a certain level or higher.

[0168] The flowcharts for the first and second decoding processes are described in detail below, based on Figure 19. In step SE1 after the start, the processor 20 causes the camera 5 to generate a read image and acquires the read image. The read image acquired in step SE1 is input to the decoding processing unit 210. In step SE2, the first decoding process is executed on the read image acquired in step SE1 without performing any image restoration. If the first decoding process is successful, the process proceeds to step SE4 to perform the termination process, i.e., the output process of the decoded result. If the first decoding process fails in step SE2, the process returns to step SE2 and the first decoding process is executed again. When a predetermined decoding timeout period has elapsed, the first decoding process for the read image is stopped.

[0169] On the other hand, in the route from step SE1 to step SE5, the decoding processing unit 210 extracts code regions for repair from the read image. Specifically, it extracts code candidate regions that are large enough to contain the code to be read as set by the tuning process, and that have a high probability of containing the code within the read image. If multiple code regions for repair are extracted at this time, they are temporarily stored as candidate regions R1, R2, ... Figure 20 shows an example of an extracted code region for repair. After that, the process proceeds to step SE6, and the value of k is set to 1.

[0170] In step SE7, a resizing process is performed on the partial image corresponding to the candidate region Rk. The resizing process reduces or enlarges the partial image corresponding to the repair code region extracted in step SE5 to a predetermined size. This resizing process allows the size of the image to be input to the neural network, described later, to be predetermined. In the example shown in Figure 20, the size of the partial image to be input to the neural network is 256 pixels × 256 pixels, but the size of the partial image is not limited to this, and it can be resized to any size considering processing speed, etc.

[0171] Step SE8 determines whether the black and white inversion setting is set to "both". In other words, image restoration is mathematically just a mapping, and it can be difficult to implement mappings from white to black, black to white, black to black, and white to white with a single neural network. Therefore, the black and white inversion process is performed based on the properties so that the image becomes black printed on a white background. Depending on how the neural network is trained, an image with white printed on a black background may also be acceptable. An example of the black and white inversion process is shown in Figure 20. Details of the black and white inversion process will be described later.

[0172] If step SE8 determines YES and the black and white inversion setting is set to "both", the process proceeds to step SE9, where the resized image is input to the image repair unit 20j and image repair is performed. Then, the process proceeds to step SE10, where the decoding processing unit 210 performs a second decoding process on the repaired image. If the second decoding process is successful in step SE10, the process proceeds to step SE4 and the termination process is performed. If the second decoding process fails in step SE10, the process proceeds to step SE11, where the resized image is subjected to a black and white inversion process. Then, in step SE11, the image after the black and white inversion process is input to the image repair unit 20j and image repair is performed (see Figure 20). Next, the process proceeds to step SE13, where the decoding processing unit 210 performs a second decoding process on the repaired image. If the second decoding process is successful in step SE13, the process proceeds to step SE4 and the termination process is performed. If the second decoding process fails in step SE13, the process proceeds to step SE14. In step SE14, it is determined whether there are still candidate regions extracted in step SE5. If there are no candidate regions, the process proceeds to step SE4 to execute the termination process.

[0173] On the other hand, if step SE8 is determined to be NO and the black and white inversion setting is not set to "both", the process proceeds to step SE15 to determine whether the black and white inversion setting is "OFF". If step SE15 is determined to be NO and the black and white inversion setting is not "OFF", the process proceeds to step SE16 to perform the black and white inversion of the resized image. Then, in step SE17, the image after the black and white inversion is input to the image repair unit 20j and image repair is performed. Next, the process proceeds to step SE18, where the decoding processing unit 210 performs a second decoding process on the repaired image. If the second decoding process is successful in step SE18, the process proceeds to step SE4 to perform the termination process. If the second decoding process fails in step SE18, the process proceeds to step SE14. In step SE14, it is determined whether there are still candidate regions, and if there are no candidate regions, the process proceeds to step SE4 to perform the termination process.

[0174] If it is determined in step SE14 that there are still candidate regions, the process proceeds to step SE19, where 1 is added to the value of k and the process proceeds to step SE7. In step SE7, since the value of k has increased by 1 since the previous flow, the same resizing process is performed on the partial image corresponding to the next candidate region Rk. Steps SE8 onwards are as described above. In other words, if decoding is successful in either the first or second decoding process, the decoding result is output at the time of success, and decoding is not performed on other candidate regions. Instead, a read image of the next workpiece W is acquired, and the same process is performed.

[0175] The post-processing unit 201 shown in Figure 20 performs a process of combining the un-repaired portion of the image after resizing with the repaired portion of the image after resizing by the image repair unit 20j. In other words, in image repair using a neural network, it is possible that the pre-repair code can be visually confirmed by a human, but the image repair is unsuccessful, or that cells become rounded after repair. To address these issues, overlapping the un-repaired portion of the image with the repaired portion in an appropriate ratio can sometimes facilitate subsequent decoding. The ratio in which the un-repaired portion of the image and the repaired portion are overlapped may be a predetermined fixed value, or it may be a value other than a fixed value. For example, it may be determined dynamically at the time of reading, or optimized through pre-evaluation (tuning), based on the stability of the repair by the image repair unit 20j. For example, in pre-evaluation, the ratio may be made adjustable by the user using a scale bar, and the adjustment may be made while presenting the combined image to the user.

[0176] (Bank switching function) In this embodiment, the parameter set storage unit 30c is configured to store a set of parameters that constitute, for example, the imaging conditions of the camera 5 and the processing conditions of the decoding processing unit 210. A parameter set can be called a bank. Multiple banks are provided, each storing different parameters. For example, the parameter set storage unit 30c stores multiple imaging conditions and coding conditions, including a first imaging condition and coding condition set by the tuning execution unit 20d, and a second imaging condition and coding condition, each as a separate parameter set.

[0177] This optical information reading device 1 is configured to allow switching between multiple parameter sets stored in the parameter set storage unit 35c, from one parameter set containing a first imaging condition and coding condition to another parameter set containing a second imaging condition and coding condition, and vice versa. Parameter set switching can be performed by the processor 20, by the user, or by a switching signal from an external control device such as a PLC 101. When the user switches the parameter sets, they can, for example, operate a parameter set switching unit integrated into the user interface. By enabling the parameter set switching unit, the parameter sets in that bank are used during operation of the optical information reading device 1. Conversely, by disabling the parameter set switching unit, the parameter sets in that bank are not used during operation of the optical information reading device 1. In other words, the parameter set switching unit is for switching from one parameter set to another.

[0178] In this example, the decoding processing unit 210, when operating with one parameter set, will switch to another parameter set and then perform image restoration and second decoding on a read image captured under different conditions if the second decoding process on the restored image fails. For example, if the decoding processing unit 210 detects that the second decoding process has failed, it will switch to a parameter set other than the one currently being used, and then perform image restoration and second decoding. If it detects that the second decoding process has also failed with this parameter set, it will switch to another parameter set that has not been used before, and then perform image restoration and second decoding. Switching parameter sets may change not only the imaging conditions of the camera 5 but also the code size of the target to be read. In this case, the size used when extracting code candidate regions that are likely to contain codes will also change. As a result, the reduction or enlargement ratio of the image input to the neural network will also change, and the results of image restoration will also change.

[0179] (Supports black and white inversion) Next, the handling of black and white inversion of codes will be explained based on Figures 21 and 22. Figure 21 illustrates the case where black and white inversion of codes is handled using two neural networks: one trained with codes printed in black on a white background, and another trained with codes printed in white on a black background. FIG. 21A shows a method in which the read image (leftmost image) is input to a neural network trained with codes printed in black on a white background and a neural network trained with codes printed in white on a black background, and the decoded result is output from each neural network. This method is called the parallel execution method. Since the read image is an image of a code printed in black on a white background, the image restored by the neural network trained with codes printed in black on a white background succeeds in decoding, but the image restored by the neural network trained with codes printed in white on a black background fails to decode.

[0180] FIG. 21B first attempts to decode the read image (leftmost image) into a neural network trained only with a code printed in white on a black background. Only if the decoding process fails is the read image input into a neural network trained with a code printed in black on a white background to generate the restored image. This method uses a different network only after the first attempt fails.

[0181] FIG.21C shows a method for selecting the neural network to input the scanned image based on user settings. The case shown on the left is when the scanned image is set to be an image of a code printed in black on a white background. In this case, the network selection module selects a neural network trained on codes printed in black on a white background. Then, the scanned image (the leftmost image) is input to the neural network trained on codes printed in black on a white background. On the other hand, the case shown on the right is when the scanned image is set to be an image of a code printed in white on a black background. In this case, the network selection module selects a neural network trained on codes printed in white on a black background. Then, the scanned image (the leftmost image) is input to the neural network trained on codes printed in white on a black background.

[0182] Figure 22 illustrates the case where only one neural network is used to handle the black and white inversion of the code. FIG. 22A shows the case where a black and white inverted image is generated by inverting the read image (leftmost image). The image without black and white inversion and the image with black and white inversion are input to a neural network trained on a code printed in black on a white background. Since the read image is an image of a code printed in white on a black background, the neural network trained on a code printed in black on a white background will fail to decode the restored image. However, by using the image after black and white inversion, the decoding process is successful.

[0183] In the example shown in FIG. 22B, the image to be read (the leftmost image) is an image of a code printed in white on a black background. Therefore, if this is input into a neural network trained on a code printed in black on a white background, the decoding process will fail. Subsequently, if an image with the colors inverted is generated and input into the neural network trained on a code printed in black on a white background, the decoding process will succeed.

[0184] In the example shown below in FIG. 22B, the image to be read (the leftmost image) is an image of a code printed in black on a white background. Therefore, if this is input into a neural network trained on a code printed in white on a black background, the decoding process will fail. Subsequently, if an image with the colors inverted is generated and input into the neural network trained on a code printed in white on a black background, the decoding process will succeed.

[0185] FIG. 22C shows a method that determines whether or not to perform black-and-white inversion processing based on user settings. The case shown at the top is when the image to be read is set to be an image of a code printed in black on a white background. In this case, the black-and-white inversion processing module does not perform black-and-white inversion processing and inputs the image directly into the neural network that has been trained with codes printed in black on a white background. On the other hand, the case shown at the bottom is when the image to be read is set to be an image of a code printed in white on a black background. In this case, the black-and-white inversion processing module performs black-and-white inversion processing. The image that has been processed with black-and-white inversion processing is then input into the neural network that has been trained with codes printed in black on a white background.

[0186] (Relationship between contrast and matching level) The matching level can be used, for example, as a score during tuning or to manage print quality / readability during operation. For instance, the matching level is defined as a positive integer value between 0 and 100, and a level of 50 or higher allows for a design that achieves a 100% readability rate when a readability test is performed. Matching level 0 represents the lowest value when the print cannot be read, and matching level 1 represents the lowest value when the print can be read.

[0187] Figure 23 is a graph showing the relationship between contrast and matching level. The solid line shows the relationship between contrast and matching level for an image restored by the image restoration unit 20j (restored image), while the dashed line shows the relationship between contrast and matching level for an image not restored by the image restoration unit 20j (read image). As is clear from this graph, the matching level of the restored image is generally higher than that of the read image, but once the image quality deteriorates beyond a certain point, the matching level drops sharply.

[0188] (Calculation process for matching level) The matching level is set for each algorithm, and generally there is one algorithm per code type. However, in this embodiment, due to the inclusion of an image repair function, two algorithms (an algorithm that repairs the image and an algorithm that does not repair the image) are executed for one code type. Of these two algorithms, the result of the one that succeeds in decoding earlier is output, so if the two algorithms succeed in decoding alternately, the matching level value may become unstable. Also, since image repair makes the image easier to read, the matching level tends to be higher than existing ones using a matching level calculation method similar to conventional methods. To address these issues, in this embodiment, the matching level value is adjusted by taking the weighted sum of the matching levels of the two algorithms. A specific example will be explained below based on the flowchart shown in Figure 24.

[0189] In step SF1 after the start, the read image is input to the decoding processing unit 210. In step SF2, the decoding processing unit 210 performs the first decoding process without image repair. In step SF3, the matching level A (MLA) based on the first decoding process is calculated. Meanwhile, in step SF4, the read image is input to the image repair unit 20j and image repair is performed. Then, the process proceeds to step SF5, where the decoding processing unit 210 performs the second decoding process on the repaired image. In step SF6, the matching level B (MLB) based on the second decoding process is calculated. Note that if decoding is unsuccessful in step SF2, MLA = 0, and if decoding is unsuccessful in step SF5, MLB = 0.

[0190] In step SF7, the values ​​of MLA and MLB are adjusted using, for example, an averaging function. Also, regardless of whether decoding is successful in step SF2 or SF5, the MLA value is weighted.

[0191] (Example of a user interface) Figure 25 shows an example of a user interface 300 displayed on the display unit 6. The processor 20 generates the user interface 300 and displays it on the display unit 6. The user interface 300 includes a first image display area 301 that displays the image currently being captured by the camera 5 (live view image), a second image display area 302 that displays the read image, and a third display area 303 that displays the restored image restored by the image restoration unit 20j. This allows the user to see the read image and the restored image in a form that allows for comparison. Furthermore, the user can understand what kind of image is being decoded, or what the image looks like after the decoding process has been performed.

[0192] (Example of an image sensor with an AI chip) Figure 26 shows an example using an image sensor 5a with an AI chip, and the image sensor 5a of the camera 5 can be configured as shown in each figure. FIG. 26A shows a case where the image sensor 5a is packaged with an AI chip that performs image repair, and after repairing the read image, it is output to the processor 20. FIG. 26B shows a case where the image sensor 5a is packaged with an AI chip that performs extraction of a repair code area and image repair, and after extracting the repair code area from the read image, the partial image corresponding to that area is repaired and then output to the processor 20. FIG. 26C shows a case where the image sensor 5a is packaged with an AI chip that performs extraction of a repair code area, and after extracting the repair code area from the read image, the partial image corresponding to that area is output to the processor 20.

[0193] Furthermore, in the example shown in FIG. 26A, the image read by the image sensor 5a may be output to the processor 20, and image repair may be performed by the AI ​​chip, with the repaired image being output to the processor 20. The same applies to the examples shown in FIG. 26B and FIG. 26C.

[0194] (Polarizing filter attachment 3) In this embodiment, as shown in Figure 2, a first illumination unit 4a and a second illumination unit 4b, each consisting of multiple light-emitting diodes, are provided as light sources that generate illumination light to illuminate the code. The polarizing filter attachment 3 includes a first polarizing plate 3a that allows light of a first polarization component to pass through from the light-emitting diodes constituting the second illumination unit 4b, and a second polarizing plate 3b that allows light of a second polarization component substantially orthogonal to the first polarization component to pass through. The first polarizing plate 3a and the second polarizing plate 3b can be provided in the shaded areas in Figure 2, for example, the first polarizing plate 3a is arranged to cover the second illumination unit 4b, and the second polarizing plate 3b is arranged to cover the optical system 5b of the camera 5 from the light incident side.

[0195] Therefore, camera 5 receives light that has passed through the first polarizing plate 3a and reflected from the surface (code) of workpiece W via the second polarizing plate 3b. As a result, the specular reflection component of workpiece W is removed, and camera 5 generates a lower-contrast reading image compared to the case without the first polarizing plate 3a and the second polarizing plate 3b. Processor 20 acquires this low-contrast reading image. Processor 20 inputs the low-contrast reading image into a neural network, converting it into a higher-contrast restored image compared to the original image, and performs a decoding process on the restored image.

[0196] In other words, by using polarizing plates 3a and 3b, the specular reflection component of the workpiece W is removed, so a read image with reduced influence from specular reflection can be obtained. Polarizing plates 3a and 3b are suitable for images with a large specular reflection component, such as when imaging a metal workpiece, but the reduced light intensity may cause the read image to become darker and the contrast to decrease. In such cases, the read performance can be improved by converting it into a high-contrast restored image through neural network inference processing.

[0197] The embodiments described above are merely illustrative in all respects and should not be interpreted restrictively. Furthermore, any modifications or changes that fall within the equivalent scope of the claims are all within the scope of the present invention. [Industrial applicability]

[0198] As described above, the optical information reading device according to the present invention can be used, for example, to read a code attached to a workpiece. [Explanation of symbols]

[0199] 1. Stationary optical information reading device 1A Handheld optical information reading device 2 Housing 2A Handheld Housing 2B Gripping part 5 Cameras 20 processors 20c Filter Processing Unit 20d Tuning Execution Unit 20e Settings Section 20f Decode Processing Unit 20g extraction part 20h Reduced part 20i Enlarged section 20j Image Restoration Department 30 Storage section 30d Neural Network Memory Unit Double job

Claims

1. An optical information reading device for reading a code attached to a workpiece, A camera that captures the code and generates a read image, A storage unit that stores the structure and parameters of a neural network that has been pre-generated by machine learning on multiple defective images having parts unsuitable for reading the code, and multiple ideal images corresponding to each of the multiple defective images, wherein the pixel resolution representing the number of pixels of one module constituting the code is within a specific range. The system includes a processor that, by inputting the read image, which has been enlarged or reduced so that the pixel resolution of the modules constituting the code in the read image generated by the camera approaches the specified range from outside the specified range to within the specified range, into a neural network composed of structures and parameters stored in the storage unit, attempts to repair the read image according to the structures and parameters stored in the storage unit, and performs a decoding process on the repaired read image, The optical information reading device is characterized in that the neural network generated by the machine learning described above has a higher repair effect on codes whose pixel resolution is within the specified range than on codes whose pixel resolution is outside the specified range.

2. In the optical information reading device according to claim 1, The structure and parameters of the neural network are pre-generated by machine learning the defective image and the ideal image, where the number of pixels of the modules constituting the code are within a specific range. The processor is an optical information reading device that inputs the read image, which has been enlarged or reduced so that the number of pixels of the modules constituting the code in the read image generated by the camera falls within the specified range, into the neural network.

3. In the optical information reading device according to claim 1 or 2, An optical information reading device further comprising a filter processing unit that performs an image processing filter on the read image generated by the camera before enlarging or reducing the read image.

4. An optical information reading device according to any one of claims 1 to 3, The optical information reading device is further equipped with a tuning execution unit that, when setting up the optical information reading device, generates multiple reading images with different magnifications or multiple reading images with different reduction ratios, inputs the generated multiple reading images into the neural network to attempt to repair each reading image, performs a decoding process on each repaired reading image to determine the reading margin, and identifies the magnification or reduction ratio of the reading image whose obtained reading margin is higher than a predetermined value as the magnification or reduction ratio to be used during operation. The processor is an optical information reading device that, when the optical information reading device is in operation, enlarges or reduces the read image to a magnification or reduction ratio specified by the tuning execution unit.

5. In the optical information reading device according to claim 4, The tuning execution unit is an optical information reading device that identifies the magnification or reduction ratio of the image with the highest reading margin among the reading margins of each read image as the magnification or reduction ratio to be used during operation.

6. An optical information reading device according to any one of claims 1 to 3, The aforementioned storage unit stores the structures and parameters of multiple neural networks with different numbers of layers or image processing filters, in association with coding conditions. The optical information reading device is further equipped with a tuning execution unit that sets code conditions included in the reading image generated by the camera when setting up the optical information reading device, reads the structure and parameters of the neural network associated with the set code conditions from the storage unit, and identifies it as the neural network to be used during operation. The processor is an optical information reading device that, when the optical information reading device is in operation, attempts to repair the read image using a neural network identified by the tuning execution unit.

7. An optical information reading device according to any one of claims 4 to 6, The tuning execution unit sets code conditions included in the image read by the camera, obtains the number of pixels of the module that is the set code condition, and if the obtained number of pixels is within the specified range, the optical information reading device does not perform the enlargement or reduction by the processor.

8. In the optical information reading device according to claim 4, The storage unit is configured to store a plurality of parameter sets with different scaling ratios by the processor or a plurality of parameter sets with different scaling ratios by the processor. The processor is an optical information reading device that performs decoding by applying any one parameter set from among a plurality of parameter sets stored in the memory unit.

9. An optical information reading device according to any one of claims 1 to 6, The optical information reading device includes a processor that searches for a code in the enlarged or reduced read image based on information for identifying the code, extracts a region containing the searched code as a code candidate region where the code is likely to exist, inputs a partial image corresponding to the extracted code candidate region into a neural network composed of structures and parameters stored in the memory, attempts to repair the partial image according to the structures and parameters stored in the memory, and performs a decoding process on the repaired partial image.

10. An optical information reading device according to any one of claims 1 to 9, An image repair unit inputs the image read by the camera into a neural network composed of the structure and parameters stored in the memory unit and performs inference processing to generate a repaired image by repairing the read image. A first decoding processing unit that performs a first decoding process on the image read by the camera, The system further comprises a second decoding processing unit that performs a second decoding process on the restored image generated by the image restoration unit, The second decoding processing unit extracts code candidate regions from the read image that are highly likely to contain a code, and sends a trigger signal to the image restoration unit to perform inference processing. When the image repair unit receives a trigger signal sent from the second decoding unit, it inputs a partial image corresponding to the code candidate region extracted by the second decoding unit into the neural network and performs an inference process to generate a repaired image by repairing the partial image. The second decoding processing unit determines grid positions indicating the positions of each cell in the code based on the restored image, and is an optical information reading device that performs decoding processing of the restored image based on the determined grid positions.

11. In the optical information reading device according to claim 10, An optical information reading device in which the inference processing by the image restoration unit and the first decoding processing by the first decoding processing unit are executed in parallel, and during the period in which the second decoding processing unit is performing the second decoding processing, there is a pause period in which the inference processing by the image restoration unit is not performed.

12. An optical information reading device according to any one of claims 1 to 11, An image repair unit inputs the image read by the camera into a neural network composed of the structure and parameters stored in the memory unit and performs inference processing to generate a repaired image by repairing the read image. A first decoding processing unit that performs a first decoding process on the image read by the camera, A second decoding processing unit which performs a second decoding process on the restored image generated by the image restoration unit, The system further includes a setting unit for setting whether or not to perform the second decoding process, The first decoding processing unit is capable of performing the first decoding process in a shorter time than the time required for the second decoding process. The setting unit is configured to set a decode timeout period longer than the time required to execute the second decode process when it is set to execute the second decode process, and to set a decode timeout period shorter than the time required to execute the second decode process when it is set not to execute the second decode process.

13. An optical information reading device according to any one of claims 1 to 9, An image repair unit inputs the image read by the camera into a neural network composed of the structure and parameters stored in the memory unit and performs inference processing to generate a repaired image by repairing the read image. A decoding processing unit that performs a first decoding process on a read image generated by the camera, and a second decoding process on a restored image generated by the image restoration unit in parallel with the first decoding process, The system further includes a setting unit for setting whether or not to perform the second decoding process, The decoding processing unit, when set by the setting unit not to execute the second decoding process, allocates the processing resources used for the second decoding process to the first decoding process, thereby speeding up the first decoding process compared to when it is set to execute the second decoding process.

14. An optical information reading device according to any one of claims 1 to 9, An image repair unit inputs the image read by the camera into a neural network composed of the structure and parameters stored in the memory unit and performs inference processing to generate a repaired image by repairing the read image. The system further comprises a decoding processing unit that performs decoding on the restored image generated by the image restoration unit, The decoding processing unit extracts code candidate regions from the read image that are highly likely to contain a code. The image repair unit inputs the partial image corresponding to the code candidate region extracted by the decoding processing unit into the neural network and performs an inference process to generate a repaired image by repairing the partial image. The decoding processing unit determines grid positions indicating the location of each cell in the code based on the repaired image, and is an optical information reading device that performs decoding of the repaired image based on the determined grid positions.

15. An optical information reading device according to any one of claims 1 to 9, An image repair unit inputs the image read by the camera into a neural network composed of the structure and parameters stored in the memory unit and performs inference processing to generate a repaired image by repairing the read image. A decoding processing unit that performs decoding on the restored image generated by the image restoration unit, The system further includes a tuning execution unit that repeatedly performs imaging and decoding by changing the imaging conditions of the camera and the decoding conditions of the decoding process, determines the optimal imaging and decoding conditions based on a matching level indicating the readability of the code calculated for each imaging and decoding condition, and performs a tuning process to set the size of the code to be read, The decoding processing unit, An optical information reading device that extracts code candidate regions from the read image that are of a size that can encompass the code to be read set by the tuning process and that have a high probability of containing the code, reduces or enlarges the extracted image to a predetermined size and inputs it into the neural network, and decodes the restored image restored by the neural network.

16. An optical information reading device according to any one of claims 1 to 9, Multiple light sources that generate illumination light to light up the code, A first polarizing plate that allows light of the first polarized component to pass through the light generated by the aforementioned light source, The system further comprises a second polarizing plate that allows light with a second polarization component substantially perpendicular to the first polarization component to pass through, The camera receives light that has passed through the first polarizing plate and reflected from the code via the second polarizing plate, and removes the specular reflection component of the workpiece, thereby generating a lower contrast reading image compared to the case without the first and second polarizing plates. The processor is an optical information reading device that inputs the low-contrast read image into the neural network, converts it into a high-contrast restored image compared to the original image, and performs a decoding process on the restored image.

17. An optical information reading device according to any one of claims 1 to 9, An image repair unit inputs the image read by the camera into a neural network composed of the structure and parameters stored in the memory unit and performs inference processing to generate a repaired image by repairing the read image. A decoding processing unit that performs decoding on the restored image generated by the image restoration unit, The system further comprises a tuning execution unit that repeatedly performs imaging and decoding by changing the imaging conditions of the camera, and sets multiple imaging conditions and code conditions for the code to be read based on a matching level indicating the readability of the code calculated for each imaging condition, The storage unit stores a plurality of imaging conditions and code conditions, including a first imaging condition and code condition and a second imaging condition and code condition set by the tuning execution unit. The optical information reading device includes an optical information reading device in which the decoding processing unit inputs a read image generated by the first imaging condition and coding condition from among a plurality of reading conditions stored in the storage unit to the neural network to generate a restored image, and if the decoding process of the restored image fails, inputs a read image generated by the second imaging condition and coding condition to the neural network to generate a restored image and performs decoding of the restored image.

18. An optical information reading device according to any one of claims 1 to 17, The camera generates a first read image containing a first code and a second read image containing a second code. The processor inputs the first and second read images, both of which have a pixel resolution outside the specified range, to the neural network after enlarging or reducing them. An optical information reading device characterized in that, if the pixel resolution of the first code falls within the specified range due to the enlargement or reduction, and the pixel resolution of the second code remains outside the specified range, the repair effect of the neural network on the first code is higher than the repair effect on the second code.

Images