Optical information reader

Abstract

To allow the repairing of a reading image and increase the efficiency of decoding even if there is a mixture of codes printed in black on a white background and codes printed in white on a black background.SOLUTION: A storage unit stores one of a first neural network for repairing a code having black cells on a white background and a second neural network for repairing a code having white cells on a black background. A decode unit executes black-white reversing processing of making the black-white relation of the reading image agree with the black-white relation that can be repaired by the neural network stored in the storage unit, and then, inputs the black-white relation into the neural network stored in the storage unit.SELECTED DRAWING: Figure 23

Description

[Technical Field]

[0001] The present disclosure relates to an optical reader that reads a code attached to a workpiece. [Background technology]

[0002] Generally, optical information readers are configured to capture an image of a code, such as a barcode or two-dimensional code, attached to a workpiece using a camera, extract and binarize the code contained in the captured image using image processing, and then decode the code to read the information. This type of optical information reader is used for the purpose of so-called traceability, which enables tracking of the distribution route of an item from the manufacturing stage to the consumption stage or disposal stage, for example.

[0003] Patent Document 1 discloses an image processing device that inputs a scanned image that has been degraded by repeated fax transmission, copying, etc., into a trained neural network to restore the image, and then decodes the restored image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-46751 A Summary of the Invention [Problem to be solved by the invention]

[0005] The image processing device in Patent Document 1 processes code images that have been degraded by fax transmission, copying, etc., and therefore is designed to process only codes printed in black on a white background, and uses a neural network that has been trained on such codes.

[0006] However, when it comes to the purpose of traceability of goods in factories and warehouses, the color of the goods is not necessarily white, but may be black or a dark color close to black. If the color of the goods is black, the color of the code printed on it will be white or a light color close to white. Also, the relationship between black and white may appear reversed depending on the way the light hits the goods.

[0007] In other words, there are workplaces where codes printed in black on a white background and codes printed in white on a black background are mixed, but a neural network that assumes codes printed in black on a white background, as in Patent Document 1, has had difficulty dealing with codes printed in white on black.

[0008] The present disclosure has been made in consideration of such points, and its purpose is to improve the decoding success rate by enabling the restoration of the read image even when a mixture of codes printed in black on a white background and codes printed in white on a black background is present. [Means for solving the problem]

[0009] To achieve the above object, in one aspect of the present disclosure, an optical information reading device includes: a memory unit that stores the structure and parameters of a neural network for estimating an ideal image corresponding to a read image generated by a camera; and a decoding unit that inputs the read image generated by the camera into a neural network configured with the structure and parameters stored in the memory unit, performs an inference process to generate a restored image by repairing the read image, and decodes the restored image. The memory unit stores either a first neural network for repairing a code having black cells on a white background or a second neural network for repairing a code having white cells on a black background. The decoding unit inverts the black-and-white relationship of the read image acquired by the camera so that it matches the black-and-white relationship that can be repaired by the neural network stored in the memory unit, and then inputs the inverted black-and-white relationship to the neural network stored in the memory unit.

[0010] According to this configuration, when the first neural network is stored, a restored image is generated by inputting the scanned image of a code having black cells on a white background directly into the first neural network, thereby improving the decoding success rate. On the other hand, when the scanned image is a code having white cells on a black background, the black and white relationship of the scanned image is inverted to create a black and white relationship that can be restored by the first neural network. Therefore, a restored image is generated by the first neural network, improving the decoding success rate.

[0011] Furthermore, if the second neural network is stored, a restored image is generated by inputting a scanned image of a code with white cells on a black background directly into the second neural network. On the other hand, if the scanned image is a code with black cells on a white background, the black-and-white relationship of the scanned image is inverted to create a black-and-white relationship that can be restored by the second neural network. Thus, a restored image is generated by the second neural network. In other words, whether the code is with black cells on a white background or white cells on a black background, a restored image can be generated by the first neural network or the second neural network, thereby improving the decoding success rate.

[0012] In another aspect, the storage unit stores a first neural network for recovering a code having black cells on a white background and a second neural network for recovering a code having white cells on a black background, and the decoding unit inputs the scanned image to at least one of the first neural network and the second neural network.

[0013] With this configuration, a restored image is generated by inputting a scanned image of a code with black cells on a white background into the first neural network, and a restored image is generated by inputting a scanned image of a code with white cells on a black background into the second neural network. Therefore, even if a code with black cells on a white background and a code with white cells on a black background are mixed, it is possible to restore the scanned image, improving the decoding success rate. [Effects of the Invention]

[0014] As described above, even if a code printed in black on a white background and a code printed in white on a black background are mixed, it is possible to restore the read image and improve the decoding success rate. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 2 is a diagram illustrating the optical information reader during operation. [Figure 2] FIG. 1 is a block diagram of an optical information reader. [Figure 3] FIG. 2 is a front view of the optical information reader. [Figure 4] FIG. 2 is a perspective view of the optical information reader as seen from the connector side. [Figure 5] FIG. 2 is a perspective view of the optical information reader as seen from the rear side. [Figure 6] FIG. 2 is a diagram illustrating each unit configured by a processor. [Figure 7] FIG. 1 is a conceptual diagram of a neural network. [Figure 8] 1 is a flowchart showing an example of a basic procedure for learning a neural network. [Figure 9] A pair of a defective image and an ideal image is shown, with FIG. 9A being an example of the defective image and FIG. 9B being an example of the ideal image. [Figure 10] FIG. 1 is a conceptual diagram of a convolutional neural network used for image conversion. [Figure 11] 10 is a flowchart showing an example of the procedure of a tuning process performed when setting up an optical information reader. [Figure 12] 10 is a flowchart illustrating an example of a decoding process procedure before a reduction ratio and an enlargement ratio are determined. [Figure 13] 10 is a flowchart illustrating an example of a decoding process procedure after a reduction ratio and an enlargement ratio are determined. [Figure 14] FIG. 14A shows an example of a scanned image generated by a camera, and FIG. 14B shows an example of an image after image processing filtering. [Figure 15] FIG. 15A shows an example of an image after reduction processing, and FIG. 15B shows an example of an image after code region extraction. [Figure 16] FIG. 10 is a diagram showing an example of restoration of a scanned image using a neural network. [Figure 17] FIG. 10 is a diagram illustrating an example of processing during operation. [Figure 18] 4 is a time chart of the decoding process by the first decoding unit and the second decoding unit. [Figure 19] 10 is a time chart of a decoding process by a decoding unit. [Figure 20] 10 is a flowchart showing an example of a first decoding process and a second decoding process. [Figure 21] 10A to 10C are diagrams illustrating example images of each process. [Figure 22] 10A and 10B are diagrams illustrating a case where a plurality of neural networks are used to deal with black and white inversion of a code. [Figure 23] FIG. 10 is a diagram illustrating a case where one neural network is used to handle black and white inversion of a code. [Figure 24] 10 is a graph showing the relationship between contrast and matching level. [Figure 25] 10 is a flowchart illustrating an example of a process for calculating a matching level. [Figure 26] FIG. 4 is a diagram illustrating an example of a user interface displayed on a display unit. [Figure 27] FIG. 10 is a diagram showing an example in which an imaging element with an AI chip is used. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present invention, its applications, or its uses.

[0017] FIG. 1 is a schematic diagram illustrating an optical information reading device 1 according to an embodiment of the present invention during operation. In this example, multiple workpieces W are placed on the upper surface of a conveyor belt B and are being transported in the direction of arrow Y in FIG. 1 . The optical information reading device 1 according to the embodiment is installed above and away from the workpieces W. The optical information reading device 1 is a code reader configured to photograph the code attached to the workpiece W and decode the code contained in the photographed image to read the information. In the example shown in FIG. 1 , the optical information reading device 1 is a stationary type. When this stationary type optical information reading device 1 is in operation, the optical information reading device 1 is fixed to a bracket or the like (not shown) to prevent movement. Note that the stationary type optical information reading device 1 may also be used in a state where it is held by a robot (not shown). Alternatively, the optical information reading device 1 may read the code on a stationary workpiece W. The stationary type optical information reading device 1 is in operation when it sequentially reads the codes on the workpieces W transported by the conveyor belt B.

[0018] In addition, a code is attached to the outer surface of each workpiece W. The code includes both a barcode and a two-dimensional code. Examples of two-dimensional codes include QR Code (registered trademark), Micro QR Code, and Data Matrix (Data code). (registered trademark) Examples of two-dimensional codes include Vericode, Aztec code, PDF417, and Maxicode. Two-dimensional codes are available in stack and matrix types, but the present invention can be applied to any two-dimensional code. The code may be attached by printing or engraving directly onto the workpiece W, or by printing on a label and then attaching it to the workpiece W; the means and method are not important.

[0019] (Read start trigger signal) The optical information reader 1 is connected to the setting device 100 and the programmable logic controller (PLC) 101 by wired connection via signal lines 100a and 101a, respectively, but this is not limiting. The optical information reader 1, the setting device 100, and the PLC 101 may each have a built-in communication module, and the optical information reader 1 may be connected to the setting device 100 and the PLC 101 wirelessly. The PLC 101 is a control device for sequentially controlling the conveyor belt B and the optical information reader 1, and a general-purpose PLC may be used. The setting device 100 may be a general-purpose or dedicated electronic computer, a portable terminal, or the like.

[0020] Furthermore, during operation, the optical information reading device 1 receives a read start trigger signal from the PLC 101 via signal line 101a, which signal specifies the start timing of code reading. The optical information reading device 1 then captures and decodes the code based on this read start trigger signal. The decoded result is then transmitted to the PLC 101 via signal line 101a. Thus, during operation of the optical information reading device 1, the read start trigger signal is repeatedly input and the decoded result is repeatedly output via signal line 101a between the optical information reading device 1 and an external control device such as the PLC 101. Note that the input of the read start trigger signal and the output of the decoded result may be performed via signal line 101a between the optical information reading device 1 and the PLC 101, as described above, or via another signal line (not shown). For example, a sensor for detecting the arrival of a workpiece W may be directly connected to the optical information reading device 1, and the read start trigger signal may be input from the sensor to the optical information reading device 1.

[0021] (Configuration of optical information reader) 2 to 5, the optical information reading device 1 includes a housing 2, an illumination unit 4, a camera 5, and a processor 20. The illumination unit 4 is a part that illuminates the workpiece W, and the camera 5 is a part that photographs the workpiece W to which a code has been applied while illuminated by the illumination unit 4, and acquires a read image including the code.

[0022] In the description of this embodiment, the up / down, left / right, and front / rear directions of the optical information reader 1 are defined as shown in Figures 3 to 5, but this is for the sake of convenience of description only and does not limit the orientation of the optical information reader 1 when in use. That is, as shown in Figure 1, the optical information reader 1 can be installed and used with its front surface (front face) facing downwards and its rear surface (rear face) facing upwards, or it can be installed and used with its front surface facing upwards, or it can be installed and used with its front surface tilted. The left / right direction of the optical information reader 1 can also be called the width direction.

[0023] The housing 2 is a generally rectangular box-like structure elongated in the vertical direction, and has at least a front surface 2a, a rear surface 2b, a left side surface 2c, a right side surface 2d, a top surface 2e, and a bottom surface 2f. The camera 5 is provided within the housing 2. As shown in FIG. 2, the camera 5 includes an imaging element 5a that captures an image of the code illuminated by the illumination unit 4, an optical system 5b including lenses, and an AF module (autofocus module) 5c. Light reflected from the code-bearing portion of the workpiece W is incident on the optical system 5b. The imaging element 5a is an image sensor consisting of a photodetector such as a CCD (charge-coupled device) or a CMOS (complementary metal oxide semiconductor) that converts the image of the code obtained through the optical system 5b into an electrical signal. The imaging element 5a is connected to the processor 20, and the electrical signal converted by the imaging element 5a is input to the processor 20. The AF module 5c is a mechanism for adjusting the position and refractive index of the focusing lens in the optical system 5b to adjust the focus. The AF module 5c is also connected to the processor 20 and is controlled by the processor 20.

[0024] 2, the illumination unit 4 has a plurality of first light-emitting elements 4a, a plurality of second light-emitting elements 4b, and a plurality of third light-emitting elements 4c. Each of the light-emitting elements 4a, 4b, and 4c is formed of, for example, a light-emitting diode. A first group is formed by the plurality of first light-emitting elements 4a, a second group is formed by the plurality of second light-emitting elements 4b, and a third group is formed by the plurality of third light-emitting elements 4c.

[0025] As shown in Figures 3 and 4, the front surface 2a of the housing 2 is provided with a light-transmitting plate 50 disposed in front of the first light-emitting element 4a, a diffusion plate 51 disposed in front of the second light-emitting element 4b, and a polarizing plate 52 disposed in front of the third light-emitting element 4c. The light-transmitting plate 50 is, for example, a colorless and transparent plate that does not have a polarizing or diffusing effect. The diffusion plate 51 is a light-transmitting plate with fine grain formed on its surface, etc., that diffuses incident light and emits it. The polarizing plate 52 is a plate that has a polarizing effect.

[0026] The optical information reading device 1 is equipped with an aimer 10 made up of a light-emitting body such as a light-emitting diode. This aimer 10 is used to indicate the field of view of the camera 5 and the position of the optical axis of the illumination unit 4 by emitting light forward of the optical information reading device 1. A user can also set up the optical information reading device 1 by referring to the light (aimer light) emitted from the aimer 10.

[0027] In this embodiment, an example has been described in which the light-transmitting plate 50, the diffuser plate 51, and the polarizer plate 52 are fixed to the front surface 2a of the housing 2. However, this is not limiting, and at least one of the light-transmitting plate 50, the diffuser plate 51, and the polarizer plate 52 may be detachably attached to the housing 2. Although not shown, for example, at least one of the light-transmitting plate 50, the diffuser plate 51, and the polarizer plate 52 may be integrated with a frame to form an attachment, and the frame may be attached to the front surface 2a of the housing 2 using, for example, a claw-fitting structure or a fastening structure using screws. In this case, at least one of the light-transmitting plate 50, the diffuser plate 51, and the polarizer plate 52 can be attached or detached as needed. The attachment may include only the diffuser plate 51, only the polarizer plate 52, or both the diffuser plate 51 and the polarizer plate 52.

[0028] 2, the illumination unit 4 has an illumination driver 4d. That is, the first light-emitting element 4a, the second light-emitting element 4b, and the third light-emitting element 4c are connected to the illumination driver 4d and controlled by the illumination driver 4d. The illumination driver 4d may be configured as part of the processor 20.

[0029] The illumination driving unit 4d can control the illumination unit 4 so that the other illumination light-emitting elements 4a and 4c are not turned on when the second light-emitting element 4b is turned on; control the illumination unit 4 so that the other illumination light-emitting elements 4b and 4c are not turned on when the first light-emitting element 4a is turned on; and control the illumination unit 4 so that the other illumination light-emitting elements 4a and 4b are not turned on when the third light-emitting element 4c is turned on. In other words, the illumination driving unit 4d is configured to switch between direct light (light transmitted through the light-transmitting plate 50), diffused light, and polarized light to illuminate the code. Furthermore, for workpieces W that do not require diffused light, switching to direct light allows a large amount of light to be irradiated onto the code, resulting in a high-contrast read image. Furthermore, switching to polarized light, if necessary, also allows a high-contrast read image to be obtained. This expands the range of workpieces W that can be read.

[0030] (main unit display) As shown in Fig. 5, a main body display unit 6 is provided on the top surface 2e of the housing 2. The main body display unit 6 is formed of, for example, an organic EL display or a liquid crystal display. As shown in Fig. 2, the main body display unit 6 is connected to the processor 20. The main body display unit 6 can display, for example, a scanned image captured by the camera 5, character strings resulting from decoding the scanned image, a scanning success rate, a matching level, and the like.

[0031] The reading success rate is the average reading success rate when the reading process is performed multiple times. The matching level is the reading margin that indicates the ease of reading a successfully decoded code. It can be calculated from the number of error corrections that occurred during decoding, and can be expressed as a number, for example. The fewer error corrections, the higher the matching level (reading margin), and conversely, the more error corrections, the lower the matching level.

[0032] (Operation buttons) A select button 11 and an enter button 12 are provided on the top surface 2e of the housing 2 and are used when setting up the optical information reader 1, etc. The select button 11 and the enter button 12 are connected to the processor 20, which is capable of detecting the operation states of the select button 11 and the enter button 12. The select button 11 is a button that is operated when selecting one option from multiple options displayed on the main body display unit 6. The enter button 12 is a button that is operated when confirming the result selected with the select button 11.

[0033] (indicator) An indicator 9 is also provided on the top surface 2e of the housing 2. The indicator 9 is connected to the processor 20 and can be configured with a light-emitting element such as a light-emitting diode. The operating status of the optical information reader 1 can be notified to the outside by the lighting state of the indicator 9. For example, the indicator 9 can be controlled so that it lights up green when the optical information reader 1 has successfully read the code image, or red when the optical information reader 1 has failed to read the code image.

[0034] (connector) A rotary connector 60 is provided at the bottom of the housing 2. The rotary connector 60 is attached to the main body of the housing 2 so as to be rotatable around a center line L3 shown in Fig. 5. The rotary connector 60 is provided with a power connector 7 to which power wiring for supplying power to the optical information reader 1 is connected, and an Ethernet connector 8 to which the setting device 100 and the PLC 130 are connected. Note that the Ethernet standard is just an example, and signal lines of standards other than the Ethernet standard can also be used.

[0035] (Specific processor configuration) 2, the processor 20 includes a CPU core 21, a DSP core 22, and an AI chip 23. The CPU core 21, the DSP core 22, and the AI ​​chip 23 are a so-called System-on-a-chip (SoC, SOC) and are mounted on a single board. Note that the DSP core 22 and the AI ​​chip 23 do not have to be an SoC, in which case they are not mounted on the same board, but this case is also within the scope of the present invention.

[0036] A high-speed RAM 41 is connected to the CPU core 21, the DSP core 22, and the AI ​​chip 23, and all of the CPU core 21, the DSP core 22, and the AI ​​chip 23 can access the RAM 41. A ROM 40 is also connected to the processor 20, and all of the CPU core 21, the DSP core 22, and the AI ​​chip 23 can access the ROM 40.

[0037] The CPU core 21 is a general-purpose processor that performs, for example, AF control, lighting control, camera control, and decoding of scanned images. The DSP core 22 performs, for example, various filter processes on scanned images. The AI ​​chip 23 is an integrated circuit dedicated to attempting to restore scanned images using a neural network, and is specialized for ultra-high-speed product-sum operations required for neural network processing.

[0038] 6, the processor 20 configures an AF control unit 20a, an imaging control unit 20b, a filter processing unit 20c, a tuning execution unit 20d, a decoding unit 20f, an extraction unit 20g, a reduction unit 20h, an enlargement unit 20i, and an image restoration unit 20j. The AF control unit 20a, the imaging control unit 20b, the filter processing unit 20c, the tuning execution unit 20d, the decoding unit 20f, the extraction unit 20g, the reduction unit 20h, and the enlargement unit 20i are configured by the arithmetic processing of the CPU core 21 or the DSP core 22. On the other hand, the image restoration unit 20j is configured by the AI ​​chip 23.

[0039] (AF control unit configuration) The AF control unit 20a is a unit that controls the AF module 5c shown in FIG. 2, and is configured to be able to focus the optical system 5b by conventionally known contrast AF or phase difference AF.

[0040] (Configuration of imaging control unit) The imaging control unit 20b is a unit that adjusts the gain of the camera 5, controls the amount of light from the illumination unit 4, and controls the exposure time (shutter speed) of the imaging element 5a. Here, the gain of the camera 5 refers to the amplification factor (also called magnification) when the brightness of the image output from the imaging element 5a is amplified by digital image processing. The amount of light from the illumination unit 4 can be changed by separately controlling the first light-emitting element 4a, the second light-emitting element 4b, and the third light-emitting element 4a. The gain, the amount of light from the illumination unit 4, and the exposure time are imaging conditions for the camera 5.

[0041] (Configuration of the filter processing unit) The filter processing unit 20c is a part that applies image processing filters to the read image. The filter processing unit 20c applies a noise removal filter that removes noise contained in the image generated by the camera 5, a contrast correction filter that corrects contrast, an averaging filter, etc. The image processing filters applied by the filter processing unit 20c are not limited to a noise removal filter, a contrast correction filter, and an averaging filter, and may include other image processing filters.

[0042] The filter processing unit 20c is configured to apply an image processing filter to a read image before image restoration, which will be described later. The filter processing unit 20c is also configured to apply an image processing filter to a read image before enlargement or reduction, which will be described later. The filter processing unit 20c may be configured to apply an image processing filter to a read image after image restoration, or may be configured to apply an image processing filter to a read image after enlargement or reduction.

[0043] (Configuration of tuning execution unit) The tuning execution unit 20d shown in FIG. 6 repeatedly captures and decodes the code while varying the imaging and decoding conditions of the camera 5. Based on a matching level indicating the readability of the code calculated under each imaging and decoding condition, the tuning execution unit 20d determines the optimal imaging and decoding conditions and sets the size of the code to be read. Specifically, the tuning execution unit 20d is a unit that, when configuring the optical information reader 1, sets various conditions (tuning parameters) to achieve optimal decoding conditions by changing imaging conditions such as the gain of the camera 5, the light intensity and exposure time of the illumination unit 4, and the image processing conditions of the filter processing unit 20c. The image processing conditions of the filter processing unit 20c include the coefficients (strength of the image processing filter), switching between image processing filters if multiple image processing filters are available, and combining different types of image processing filters. Appropriate imaging and image processing conditions vary depending on factors such as the effect of external light on the workpiece W during transport and the color and material of the surface on which the code is attached. Therefore, the tuning execution unit 20d searches for more appropriate imaging conditions and image processing conditions, and sets the processing to be performed by the AF control unit 20a, the imaging control unit 20b, and the filter processing unit 20c.

[0044] The tuning execution unit 20d is configured to be able to acquire the size of a code included in a scanned image generated by the camera 5. To acquire the code size, the tuning execution unit 20d first searches for the code based on a feature that indicates its likelihood of being a code. Next, the tuning execution unit 20d acquires the PPC, code type, code size, etc. of the searched code. The PPC, code type, code size, etc. are included in the code parameters or code conditions, and therefore the tuning execution unit 20d is configured to be able to acquire the code parameters or code conditions of the searched code.

[0045] A two-dimensional code is composed of multiple white and black modules arranged randomly. For example, in QR Code Model 2, the number of modules ranges from 21 x 21 to 177 x 177. The optical information reader 1 reads the code to be read in advance, limiting the number of modules and PPC, and the code size (pixel size) is limited to the number of modules x PPC.

[0046] When focusing on one of the modules that make up a code, the PPC is a code parameter that indicates how many pixels (picture elements) that module is made up of. After identifying one module, the tuning execution unit 20d can obtain the PPC by counting the number of pixels that make up that module.

[0047] The code type refers to the type of code, such as QR code, data matrix code, VeriCode, etc. Since each code has its own characteristics, tuning execution unit 20d can distinguish the code type, for example, based on the presence or absence of a finder pattern.

[0048] The code size can be calculated from the number of modules arranged vertically or horizontally in the code and the PPC. The tuning execution unit 20d calculates the number of modules arranged vertically or horizontally in the code, calculates the PPC, and obtains the code size by multiplying the number of modules arranged vertically or horizontally by the PPC.

[0049] (Decoder configuration) The decoding unit 20f decodes the read image obtained by the camera 5 when the code is irradiated with light from the first light-emitting element 4a through the light-transmitting plate 50, the read image obtained by the camera 5 when the code is irradiated with light from the second light-emitting element 4b through the diffusion plate 51, and the read image obtained by the camera 5 when the code is irradiated with light from the third light-emitting element 4c through the polarizing plate 52.

[0050] The decoding unit 20f decodes the black and white binary data. A table showing the correspondence between the coded data can be used for decoding. Furthermore, the decoding unit 20f checks whether the decoded result is correct according to a predetermined check method. If an error is found in the data, an error correction function is used to calculate the correct data. The error correction function differs depending on the type of code.

[0051] In this embodiment, the decoding unit 20f decodes the code contained in the read image after image restoration (restored image), which will be described later, but can also decode the code contained in the read image before image restoration. The decoding unit 20f is configured to write the decoding result obtained by decoding the code to the decoding result storage unit 30b of the storage unit 30 shown in Figure 2.

[0052] (Configuration of extraction unit) The extraction unit 20g extracts code candidate areas from the scanned image generated by the camera 5 that are likely to contain a code. The code candidate areas can be extracted based on features that indicate the likelihood of a code. In this case, the features that indicate the likelihood of a code serve as information for identifying the code. For example, the extraction unit 20g can acquire a scanned image and search for a code in the acquired scanned image based on the features that indicate the likelihood of a code. Specifically, the extraction unit 20g searches the acquired scanned image for a portion that has a predetermined or higher level of features that indicate the likelihood of a code. If a portion with a feature that indicates the likelihood of a code is found, the extraction unit 20g extracts the portion as a code candidate area. The code candidate area may include areas other than codes, but it must at least include a portion that is more likely to be a code than a predetermined level. Note that the code candidate area is merely an area where a code is likely to exist, and as a result, it may not contain a code.

[0053] (Configuration of storage unit) The storage unit 30 shown in FIG. 2 can be configured as a readable / writable storage device such as an SSD (Solid State Drive). However, the following storage units 30a, 30b, 30c, and 30d may be provided in a ROM 40 instead of the storage device. That is, this embodiment also covers a configuration in which the image data storage unit 30a, the decoded result storage unit 30b, the parameter set storage unit 30c, and the neural network storage unit 30d are included in the ROM 40. The image data storage unit 30a stores the scanned image generated by the camera 5. The decoded result storage unit 30b stores the decoded result of the code executed by the decoding unit 20f. The parameter set storage unit 30c stores the results of tuning executed by the tuning execution unit 20d, various conditions set by the tuning execution unit 20d, and various conditions set by the user. The neural network storage unit 30d stores the structure and parameters of a trained neural network, which will be described later.

[0054] (Image restoration using neural networks) The optical information reading device 1 has an image restoration function that uses a neural network to restore a read image acquired by the camera 5. In this embodiment, instead of performing machine learning in the optical information reading device 1, the optical information reading device 1 is configured to store in advance the structure and parameters of a neural network that has already completed learning, and to perform image restoration using a neural network configured with the stored structure and parameters.

[0055] 7, the neural network has an input layer to which input data (image data in this example) is input, an output layer that outputs output data, and an intermediate layer provided between the input layer and the output layer. For example, multiple intermediate layers can be provided, thereby making it possible to create a neural network with a multi-layer structure.

[0056] (Neural network training) First, the basic procedure for training a neural network will be described with reference to the flowchart shown in Fig. 8. Training of the neural network can be performed using a computer prepared for training purposes other than the optical information reading device 1, but it may also be performed using a general-purpose computer other than that for training. Furthermore, the neural network training method may be a conventionally known method.

[0057] After starting, in step SA1, data of a previously prepared defective image is read. A defective image is an image that has parts that are inappropriate for reading a code, and an example of such an image is shown in FIG. 9A. An inappropriate part is, for example, a dirty part or a colored part. Then, proceeding to step SA2, data of a previously prepared ideal image is read. An ideal image is an image that is appropriate for reading a code, and an example of such an image is shown in FIG. 9B. A defective image and an ideal image are paired, and multiple such pairs are prepared.

[0058] In step SA2, a loss function is calculated to determine the difference between the defective image and the ideal image. In step SA3, the neural network parameters are updated to reflect the difference determined in step SA2.

[0059] In step SA4, it is determined whether the machine learning completion condition is met. The machine learning completion condition can be set based on the difference calculated in step SA2. For example, if the difference is equal to or less than a predetermined value, it can be determined that the machine learning completion condition is met. If step SA4 returns YES and the machine learning completion condition is met, the machine learning is terminated. On the other hand, if step SA4 returns NO and the machine learning completion condition is not met, the process returns to step SA1, where another defective image is read, and then the process proceeds to step SA2, where another ideal image paired with the other defective image is read.

[0060] In this way, a trained neural network can be generated in advance by performing machine learning on multiple defective images and multiple ideal images corresponding to the multiple defective images. By storing the structure and parameters of the trained neural network in the optical information reading device 1, a trained neural network can be constructed within the optical information reading device 1. In other words, when performing machine learning, a huge number of defective images and ideal images are input into the neural network, which requires extremely high computing power. It is difficult to ensure such high computing power within the optical information reading device 1, as this may result in the device not being able to meet the demand for a compact and lightweight device. As in this example, by performing machine learning outside the optical information reading device 1 and storing the structure and parameters of the resulting neural network within the optical information reading device 1, image restoration using a neural network can be achieved while realizing a compact and lightweight optical information reading device 1.

[0061] Figure 10 is a conceptual diagram of a convolutional neural network (CNN) configured as described above. "Convolution" extracts the features of the input image. This layer is composed of convolution operations similar to image processing filters. The filter weights are called kernels, and feature extraction is performed according to the kernels. Convolution layers generally have multiple different kernels, and the number of maps (D) increases according to the number of kernels. The kernel size can be, for example, 3x3 or 5x5.

[0062] "Pooling" performs a reduction process to combine the responses of each kernel. "Deconvolution" uses a deconvolution filter to reconstruct an image from the feature values. "Unpooling" performs an expansion process to sharpen the response values.

[0063] The structure and parameters of the trained neural network are stored in the neural network storage unit 30d of the storage unit 30 shown in Fig. 2. The structure of the neural network refers to the number of intermediate layers (D) provided between the input layer and the output layer, the number of image processing filters, etc. The parameters of the neural network are the parameters set in step SA3 of the flowchart shown in Fig. 7.

[0064] The neural network storage unit 30d can store the structures and parameters of multiple neural networks with different numbers of layers or image processing filters. Specifically, the neural network storage unit 30d can store both a first neural network for repairing code with black cells on a white background and a second neural network for repairing code with white cells on a black background.

[0065] A code with black cells on a white background is a code printed in black on a white background. When training the first neural network, pairs of defective and ideal images of a code with black cells on a white background are used.

[0066] In addition, a code with white cells on a black background is a code printed in white on a black background. When training the second neural network, pairs of defective and ideal images of a code with white cells on a black background are used.

[0067] The neural network storage unit 30d may store the structure and parameters of either the first neural network or the second neural network.

[0068] Furthermore, the neural network storage unit 30d can store the structure and parameters of multiple neural networks in association with their corresponding chord conditions. For example, the structure and parameters of a neural network for constructing a certain neural network can be associated with the corresponding chord conditions. Since the optimal number of layers or filters of a neural network can be determined in advance by a first chord condition, particularly a first PPC, the association is performed so as to maintain this relationship. Similarly, the structure and parameters of another neural network can be associated with a second chord condition that is different from the first chord condition.

[0069] If the neural network storage unit 30d stores the structures and parameters of multiple neural networks, the tuning unit 20d can be configured to operate as follows when setting the optical information reading device 1. That is, when setting the code conditions contained in the read image generated by the camera 5, the tuning unit 20d reads the structure and parameters of the neural network associated with the set code conditions from the neural network storage unit 30d and identifies it as the neural network to be used during operation. For example, if the tuning unit 20d sets a first code condition as the code condition contained in the read image, it reads the structure and parameters of the neural network associated with the first code condition from the neural network storage unit 30d. Then, it identifies the structure and parameters of the neural network as the neural network to be used during operation of the optical information reading device 1. This allows the read image to be repaired using the structure and parameters of the neural network that are optimal for the code condition, thereby improving reading accuracy.

[0070] When training the neural network, it is also possible to train the neural network on defective images and ideal images whose pixel resolution of the modules constituting the code is within a specific range. The pixel resolution of the module can be expressed, for example, by the number of pixels (PPC) of one module constituting the code, and the neural network may be trained on only defective images and ideal images whose PPC is within a specific range.

[0071] The specific range includes pixel resolutions that provide a high level of restoration effect on scanned images, but excludes pixel resolutions that provide little or no improvement in restoration effect on scanned images, and can be set on the premise of restoring images that contain codes made up of multiple modules. This allows for an improved processing speed as a neural network structure specialized for restoring images that contain codes.

[0072] The specific range of pixel counts can be, for example, 4 PPC or more and 6 PPC or less, but it may also be less than 4 PPC or more than 6 PPC. Also, by limiting the specific range to 4 PPC or more and 6 PPC or less for machine learning, it is possible to eliminate unnecessary scale variations from the images used for learning and optimize the trained neural network structure.

[0073] (Repair attempt) As shown in Fig. 6, the processor 20 of the optical information reading device 1 is provided with an image restoration unit 20j configured with an AI chip 23. The image restoration unit 20j reads out the structure and parameters of a neural network stored in the neural network storage unit 30d of the storage unit 30, and configures a neural network in the optical information reading device 1 using the read structure and parameters of the neural network. The neural network configured in the optical information reading device 1 is the same as the trained neural network trained according to the procedure shown in the flowchart of Fig. 8.

[0074] The image restoration unit 20j inputs the scanned image generated by the camera 5 into a neural network, and attempts to restore the scanned image in accordance with the structure and parameters of the neural network stored in the neural network storage unit 30d. Then, the decoding unit 20f executes a decoding process on the restored scanned image.

[0075] The filter processing unit 20c is configured to apply an image processing filter to the read image generated by the camera 5 before the image restoration unit 20j inputs the read image to the neural network. This allows appropriate image processing to be performed before attempting restoration using the neural network, resulting in more accurate restoration of the read image.

[0076] The image restoration unit 20j may attempt image restoration for all read images decoded by the decoding unit 20f, or may attempt image restoration for only some of the read images decoded by the decoding unit 20f. For example, the decoding unit 20f performs a decoding process on the read image before attempting restoration and determines whether the decoding was successful. If the decoding is successful, it means that the read image does not require restoration, and the result is output as is. On the other hand, if the decoding fails, the image restoration unit 20j attempts to restore the read image, and the decoding unit 20f performs a decoding process on the restored read image.

[0077] (Input partial image) The image input by the image restoration unit 20j to the neural network may be the entire scanned image generated by the camera 5, but the larger the size of the input image, the greater the amount of calculation by the neural network, which increases the calculation load on the processor 20 and may ultimately result in a decrease in processing speed. Specifically, in an FCN (Fully Convolutional Network) type neural network as shown in Fig. 10, assuming the structure is the same, the amount of calculation is proportional to the input image size, and since the input image size is proportional to the square of the code size, the amount of calculation is also proportional to the square of the code size.

[0078] On the other hand, there are virtually no cases where the code is present in the entire scanned image generated by camera 5. During normal operation, the code is present only in a small area of ​​the scanned image, and if only that area can be repaired using a neural network, reading accuracy can be improved.

[0079] Therefore, the image input by the image restoration unit 20j to the neural network can be a part of the scanned image generated by the camera 5, i.e., a partial image including a code. Specifically, the image restoration unit 20j cuts out a partial image corresponding to the code candidate area extracted by the extraction unit 20g from the scanned image generated by the camera 5. The image restoration unit 20j inputs the cut-out partial image to the neural network, and attempts to restore the partial image according to the structure and parameters stored in the neural network storage unit 30d. The decoding unit 20f then performs decoding processing on the restored partial image.

[0080] As mentioned above, it is rare for a code to be present in the entire scanned image, so the size of the partial image corresponding to the code candidate area is smaller than the size of the scanned image. This reduces the size of the image input to the neural network, reducing the computational load on the processor 20 and ultimately achieving faster processing speeds. Furthermore, because the partial image corresponds to an area where a code is likely to be present, it is possible to create an image that includes the information necessary for reading, i.e., the entire code. Because this image including the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

[0081] The image restoration unit 20j can determine the input size of the partial image to the neural network based on the code size acquired by the tuning execution unit 20d. As described above, the tuning execution unit 20d can acquire the code size. The image restoration unit 20j increases the input size of the partial image to be input to the neural network by a predetermined amount compared to the code size acquired by the tuning execution unit 20d.

[0082] That is, while inputting a partial image into the neural network can reduce the computational load, if the code is rotated or the position of the code cannot be accurately detected, a portion of the code in the partial image may be missing, potentially resulting in a failure in the decoding process. In this embodiment, the input size to the neural network is not the same as the code size acquired by the tuning execution unit 20d, but is larger than the code size by a predetermined amount, thereby preventing a portion of the code from being missing from the partial image. By making the input size larger than the code size by a predetermined amount, an upper limit can be placed on the input size to the neural network, thereby preventing the computational load from becoming too heavy. The "predetermined amount" may mean, for example, that the horizontal (or vertical) length of the partial image input to the neural network is 1.5 times or more, 2.0 times or more, or 3.0 times or more the horizontal (or vertical) length of the code acquired by the tuning execution unit 20d.

[0083] When the image restoration unit 20j determines the input size of the partial image to be input to the neural network, it can determine the size based on the number of pixels in one module that makes up the code acquired by the tuning execution unit 20d and the number of modules arranged vertically or horizontally in the code.

[0084] Furthermore, the tuning execution unit 20d can set the code conditions as described above. The image restoration unit 20j can also set the size of the scanned image to be input to the neural network based on the code conditions set by the tuning execution unit 20d. For example, when the image restoration unit 20j acquires the code size from the code conditions set by the tuning execution unit 20d, it inputs a partial image that is a predetermined amount larger than the acquired code size to the neural network. If the code size is larger, the size of the partial image to be input to the neural network will be larger, and if the code size is smaller, the size of the partial image to be input to the neural network will be smaller. In other words, the size of the partial image to be input to the neural network can be changed according to the code conditions.

[0085] (Zoom in / out function) In order for a convolutional neural network like the one shown in Figure 10 to fully capture the features of the code and restore it to an image suitable for decoding, it is necessary to extract features within a range covering a certain number of modules. For example, if a feature value obtained from the neural network is a value extracted from a 6x6 module range, and an image is captured with one module consisting of 20 pixels, the neural network must be designed to calculate a feature value aggregated from a 120x120 pixel range. In other words, the wider the pixel range to be covered, the deeper the neural network hierarchy must be. For example, to aggregate feature values ​​from the above-mentioned 120x120 pixel range, six convolutional layers are required.

[0086] However, since the code is composed of a random arrangement of white and black modules, the pixel values ​​have a weak correlation over a wide range, and expanding the pixel range beyond a certain point hardly improves the restoration effect. Such features are called narrow-range features in this specification.

[0087] Furthermore, the arrangement of fixed module patterns, such as finder patterns, contained in code, or the rectangular shape of the code as a whole can be considered wide-range features, but if the module and background are simply roughly separated, repair is sufficient using only narrow-range features, without using wide-range features. Wide-range features can be defined as features that have a fixed shape over a wide range.

[0088] Therefore, when training the neural network shown in Figure 8, it is possible to use images in which the PPC of the code is within a specific range. While this can improve the processing speed as a neural network structure specialized for repairing images containing codes, there is a concern that the repair effect for scanned images in which the PPC is outside the specific range may be reduced. Note that when training the neural network, it is also possible to use images in which the PPC of the code is outside the specific range.

[0089] The optical information reader 1 of this embodiment is provided with a function for reducing and enlarging a read image so that the PPC of the read image falls within a specific range. As shown in Fig. 6, the processor 20 is configured with a reduction unit 20h and an enlargement unit 20i. The reduction unit 20h is a part that generates a reduced read image so that the pixel resolution of the modules that make up the code in the read image generated by the camera 5 falls within the above-mentioned specific range, and the enlargement unit 20i is a part that generates an enlarged read image so that the pixel resolution of the modules that make up the code in the read image generated by the camera 5 falls within the above-mentioned specific range.

[0090] The reduction unit 20h and the enlargement unit 20i determine whether the PPC of the code in the scanned image generated by the camera 5 is outside a specific range (4 PPC to 6 PPC), and if it is within the specific range, they do not perform reduction or enlargement, but if it is outside the specific range, they perform reduction or enlargement. By performing reduction or enlargement, the PPC of the code in the partial image input to the neural network by the image restoration unit 20j falls within the specific range.

[0091] The image restoration unit 20j inputs the scanned image, enlarged or reduced to fit within a specific range, into a neural network configured with the structure and parameters stored in the neural network storage unit 30d, and attempts to restore the scanned image according to the structure and parameters. The decoding unit 20f then performs a decoding process on the restored scanned image.

[0092] The extraction unit 20g may be configured to search for a code in the scanned image enlarged by the enlargement unit 20i or reduced by the reduction unit 20h, and extract an area containing the searched code as a code candidate area where a code is likely to exist. In this case, the image restoration unit 20j inputs a partial image corresponding to the code candidate area extracted by the extraction unit 20g into a neural network configured with the structure and parameters stored in the neural network storage unit 30d, and attempts to restore the scanned image in accordance with the structure and parameters.

[0093] When setting up the optical information reading device 1, the tuning execution unit 20d can generate multiple read images (enlarged images) with different magnification ratios and multiple read images (reduced images) with different reduction ratios. The tuning execution unit 20d inputs the generated multiple read images (enlarged images or reduced images) into a neural network to attempt to repair each read image, and performs a decoding process on each repaired read image to determine a reading margin that indicates the ease of reading the code. The tuning execution unit 20d identifies the magnification or reduction ratio of the read image for which the determined reading margin is higher than a predetermined value as the magnification or reduction ratio to be used during operation. When the tuning execution unit 20d identifies the magnification or reduction ratio, during operation of the optical information reading device 1, the reduction unit 20h and the enlargement unit 20i enlarge or reduce the read image at the magnification or reduction ratio identified by the tuning execution unit 20d.

[0094] That is, for example, if the specific range has a certain width, it is conceivable that the reading margin will change if the magnification or reduction ratio is changed even within that specific range. Since the magnification or reduction ratio of the read image during operation can be specified so that the reading margin calculated by the tuning execution unit 20d is higher than a predetermined value, processing speed and reading accuracy are improved.

[0095] After determining the reading margin of each scanned image, the tuning execution unit 20d can specify the magnification or reduction ratio of the scanned image with the highest reading margin among the determined reading margins as the magnification or reduction ratio to be used during operation, thereby further improving processing speed and scanning accuracy.

[0096] Furthermore, by utilizing the ability of the reduction unit 20h and the enlargement unit 20i to reduce and enlarge the scanned image, the number of parameter sets can be increased. Multiple parameter sets with different reduction ratios by the reduction unit 20h and multiple parameter sets with different enlargement ratios by the enlargement unit 20i can be generated, and these parameter sets can be stored in the parameter set storage unit 30c of the storage unit 30. For example, if multiple reduction ratios by the reduction unit 20h can be set, such as 1 / 2, 1 / 4, and 1 / 8, the parameter set with a reduction ratio of 1 / 2, a parameter set with a reduction ratio of 1 / 4, and a parameter set with a reduction ratio of 1 / 8 can be stored in the parameter set storage unit 30c. Furthermore, if multiple enlargement ratios by the enlargement unit 20i can be set, such as 2x, 4x, and 8x, the parameter set with a magnification ratio of 2x, a parameter set with a magnification ratio of 4x, and a parameter set with a magnification ratio of 8x can be stored in the parameter set storage unit 30c. Any one of the multiple parameter sets stored in the parameter set storage unit 30c can then be applied.

[0097] (Image restoration filter) The image restoration unit 20j may be a part that executes an image restoration filter that attempts to restore the scanned image using a neural network. The image restoration filter is a filter that attempts to restore the scanned image according to the structure and parameters stored in the neural network storage unit 30d by inputting the scanned image to a neural network configured with the structure and parameters stored in the neural network storage unit 30d, and performs the same function as the image restoration described above.

[0098] When the image restoration filter is executable, a setting unit 20e (shown in FIG. 6) for setting the image restoration filter can be provided. The setting unit 20e is configured to be able to accept the setting of the image restoration filter by the user. For example, when setting up the optical information reading device 1, the setting unit 20e can be configured to generate a user interface that allows the user to select either "apply image restoration filter" or "do not apply image restoration filter," display the user interface on the display unit 6, and accept the user's selection. Therefore, the image restoration unit 20j attempts to restore the read image by applying the image restoration filter set by the setting unit 20e to the read image.

[0099] The setting unit 20e may be included in the tuning execution unit 20d, or may be configured separately from the tuning execution unit 20d. When the setting unit 20e is included in the tuning execution unit 20d, it becomes possible to set parameters related to the image restoration filter during tuning. The parameters related to the image restoration filter can include "application of image restoration filter" and "non-application of image restoration filter." "Application of image restoration filter" means setting the image restoration filter to be executable, and "non-application of image restoration filter" means not setting the image restoration filter.

[0100] The parameters related to the image restoration filter are also information related to the image restoration filter. In this case, a parameter set including information related to the image restoration filter settings can be stored in the parameter set storage unit 30c. The parameters related to the image restoration filter include "apply image restoration filter" and "not apply image restoration filter." Therefore, the parameter sets stored in the parameter set storage unit 30c include a first parameter set that enables the image restoration filter and a second parameter set that does not set the image restoration filter. When the optical information reading device 1 is in operation, one of the first parameter set and the second parameter set is applied. When the first parameter set is applied, the decoding unit 20f performs a decoding process on a read image on which image restoration has been performed using the image restoration filter. Meanwhile, when the second parameter set is applied, the decoding unit 20f performs a decoding process on a read image on which image restoration has not been performed.

[0101] The parameter sets stored in the parameter set storage unit 30c also include items for setting code conditions included in the scanned image generated by the camera 5. The items for setting code conditions include the PPC, code type, code size, etc. acquired by the tuning execution unit 20d. For example, one parameter set may include the PPC, code type, and code size as items for setting code conditions, the gain of the camera 5, the light intensity and exposure time of the illumination unit 4 as imaging conditions, and the type of image processing filter, parameters of the image processing filter, and application items of the image restoration filter as items of the image processing filter applied by the filter processing unit 20c. The values ​​set by the tuning execution unit 20d may be used as they are, or the user may change them as desired.

[0102] (An example of tuning process steps) An example of the procedure of the tuning process performed by the tuning execution unit 20d when setting up the optical information reader 1 will be specifically described with reference to the flowchart shown in Fig. 11. In step SB1 after the start of the flowchart shown in Fig. 11, the tuning execution unit 20d controls the illumination unit 4 and camera 5 to cause the camera 5 to generate a read image, and the tuning execution unit 20d acquires the read image. At this time, the presence and type of an image processing filter to be executed before the decoding process and the decoding process parameters for applying image restoration by a neural network are set to arbitrary parameters. Next, proceeding to step SB2, the tuning execution unit 20d causes the decoding unit 20f to execute decoding process on the acquired read image.

[0103] After the decoding process, the process proceeds to step SB3, where the tuning execution unit 20d determines whether the decoding process of step SB2 was successful. If the determination in step SB3 is NO and the decoding process of step SB2 has failed, that is, if the code cannot be read, the process proceeds to step SB4, where the decoding process parameters are changed to other parameters, and the decoding process is executed again in step SB2. If the decoding process has failed with all of the decoding process parameters, this flow ends and the user is notified.

[0104] On the other hand, if the determination in step SB3 is YES and the decoding process in step SB2 is successful, the process proceeds to step SB5, where the tuning execution unit 20d determines code parameters (PPC, code type, code size, etc.). After the tuning execution unit 20d determines the code parameters, the structure and parameters of the neural network stored in the neural network storage unit 30d in association with the code parameters are also determined (step SB6). In step SB6, a neural network is configured using the structure and parameters read from the neural network storage unit 30d.

[0105] In step SB7, the image restoration unit 20j attempts to restore the read image by inputting the read image into the neural network configured in step SB6, and the decoding unit 20f executes decoding processing on the restored read image. Thereafter, the process proceeds to step SB8, where the tuning execution unit 20d evaluates the reading margin based on the decoding processing result of step SB7 and temporarily stores the evaluation result.

[0106] In step SB9, it is determined whether the decoding process has been completed for all the decoding process parameters. If the determination in step SB9 is NO, meaning that the decoding process has not been completed for all the decoding process parameters, the process proceeds to step SB10, where the decoding process parameters are changed to other parameters, and then the process proceeds to step SB7.

[0107] On the other hand, if step SB9 returns YES and the decoding process is completed for all the decoding process parameters, the process proceeds to step SB11. In step SB11, the tuning execution unit 20d selects the decoding process parameter with the highest read margin from among all the decoding process parameters, and determines the selected decoding process parameter as the parameter to be applied during operation. Note that during the tuning process, the imaging conditions and the like are also set to appropriate conditions.

[0108] (Decoding procedure before determining reduction and enlargement ratios) Next, an example of a decoding process procedure before determining the reduction ratio and enlargement ratio will be specifically described with reference to the flowchart shown in Fig. 12. The process specified in the flowchart shown in Fig. 12 can be executed in step SB2 of the flowchart shown in Fig. 11.

[0109] 12, in step SC1 after the start, the tuning execution unit 20d selects an arbitrary image processing filter from among a plurality of image processing filters. This image processing filter is a filter to be executed by the filter processing unit 20c. Then, in step SC2, the filter processing unit 20c executes the image processing filter selected in step SC1 on the scanned image.

[0110] Next, the process proceeds to step SC3, where an image pyramid is created. The image pyramid is composed of the original read image, an image obtained by reducing the original read image by half, an image obtained by reducing the original read image by a quarter, an image obtained by reducing the original read image by an eighth, and so on. The reduction of the read image is performed by a reduction unit 20h. The image pyramid can also be composed of the original read image, an image obtained by enlarging the original read image by two times, an image obtained by enlarging the original read image by four times, an image obtained by enlarging the original read image by eight times, and so on. The enlargement of the read image is performed by an enlargement unit 20i. Note that either the reduced image or the enlarged image may be omitted.

[0111] After creating the image pyramid, the process proceeds to step SC4, where an arbitrary scanned image is selected from the multiple scanned images that make up the image pyramid. The selected images may include the original scanned image that has not been reduced or enlarged. In step SC5, the extraction unit 20g extracts a code candidate area that is likely to contain a code from the scanned image selected in step SC4. Then, the process proceeds to step SC6, where the image restoration unit 20j inputs the partial image corresponding to the code candidate area extracted in step SC5 into a neural network and attempts to restore the scanned image. The size of the partial image to be input into the neural network is determined by the code conditions described above.

[0112] After the scanned image is restored, the process proceeds to step SC7, where a positioning process is performed on the outline of the code and the modules that make up the code. After the positioning process, the process proceeds to step SC8, where it is determined whether each module that makes up the code is white or black. After determining whether it is black or white, the process proceeds to step SC9, where the decoding unit 20f performs a decoding process on the restored scanned image. In the decoding process, for example, a method of restoring a character string from a 0-1 matrix of the modules can be used.

[0113] Thereafter, the process proceeds to step SC10, where the tuning execution unit 20d determines whether the decoding process of step SC9 was successful. If the determination in step SC10 is NO and the decoding process of step SC9 has failed, the process proceeds to step SC11, where it is determined whether all the read images constituting the image pyramid have been selected. If the determination in step SC11 is NO and all the read images constituting the image pyramid have not been selected, the process proceeds to step SC4, where another reduced read image or another enlarged read image constituting the image pyramid is selected, and the process proceeds to step SC5.

[0114] On the other hand, if the determination in step SC11 is YES and all the scanned images constituting the image pyramid have been selected, this flow ends and the conditions under which the decoding was successful are stored.

[0115] (Decoding procedure after determining reduction and enlargement ratios) Next, an example of the decoding process procedure after determining the reduction ratio and enlargement ratio will be specifically described with reference to the flowchart shown in Fig. 13. The process specified in the flowchart shown in Fig. 13 can be executed in step SB7 of the flowchart shown in Fig. 11 and when the optical information reader 1 is in operation.

[0116] In step SD1 after the start of the flowchart shown in Fig. 13, the filter processing unit 20c applies the image processing filter selected in step SC1 of the flowchart shown in Fig. 12 to the read image. At this time, if the filter processing unit 20c applies an averaging filter to the read image shown in Fig. 14A of Fig. 14, an image after application of the image processing filter as shown in Fig. 14B of Fig. 14 is obtained.

[0117] Then, the process proceeds to step SD2, where the scanned image is reduced or enlarged as necessary. Specifically, if the PPC of the code in the scanned image after the image processing filter is applied is within a specific range, reduction or enlargement is not performed. If it is outside the specific range, reduction or enlargement is performed so that the PPC falls within the specific range. FIG. 15A in FIG. 15 shows an example of a scanned image after reduction processing.

[0118] Next, in step SD3, the extraction unit 20g extracts a code candidate area that is likely to contain a code from the scanned image reduced or enlarged in step SD2. FIG. 15A in Fig. 15 shows an example of an extracted code candidate area image. Note that if the scanned image was not reduced or enlarged in step SD2, the extraction process is performed on the original scanned image in step SD3.

[0119] Then, the process proceeds to step SD4, where the image restoration unit 20j inputs the partial image corresponding to the code candidate region extracted in step SD3 into a neural network and attempts to restore the scanned image. FIG. 16 shows examples of the scanned image before and after restoration. After the scanned image is restored, steps SD5 to SD7 are performed, and this flow ends. Steps SD5 to SD7 are the same as steps SC7 to SC9 in the flowchart shown in FIG. 12.

[0120] (Example of processing during operation) 17 is a diagram illustrating an example of a processing procedure during operation of the optical information reader 1. The processor 20 configures a pre-processing unit 200, a post-processing unit 201, a first decoding unit 202, and a second decoding unit 203. The first decoding unit 202 and the second decoding unit 203 are included in the decoding unit 20f, and are capable of executing decoding processes in parallel. The pre-processing unit 200 is a part that applies a noise reduction filter, a contrast correction filter, an averaging filter, etc. to the read image generated by the camera 5.

[0121] First decoding unit 202 is a part that executes decoding processing on the scanned image generated by camera 5, and the decoding processing by first decoding unit 202 is also referred to as first decoding processing. When processor 20 has multiple cores (N cores), first decoding unit 202 is configured by cores 0 to i. First decoding unit 202 may also be configured by a single core.

[0122] The second decoding unit 203 is a part that executes a decoding process on the restored image generated by the image restoration unit 20j, and the decoding process by the second decoding unit 203 is also referred to as a second decoding process. If the processor 20 has N cores, the second decoding unit 203 is configured by core i+1 to core N-1. The second decoding unit 203 may also be configured by a single core.

[0123] 18, when the first decoding unit 202 is made up of cores 0 to i, each of cores 0 to i extracts a code candidate area where a code is likely to exist from the scanned image generated by camera 5, performs positioning processing of the code outline and the modules that make up the code, and then determines whether each module that makes up the code is white or black and performs decoding processing. The processing result is output to the post-processing unit 201.

[0124] On the other hand, when the second decoding unit 203 is made up of cores i+1 to N-1, each of cores i+1 to N-1 extracts a code candidate area from the read image that is likely to contain a code as a repair code area, and sends a trigger signal to the image repair unit 20j to execute inference processing. Although Fig. 18 shows one task being executed by two cores, one task may be executed by any one or more cores.

[0125] Specifically, when cores N-2 and N-1 constituting the second decoding unit 203 extract a repair code candidate area, they output a trigger signal (Trigger 1) to the image restoration unit 20j, causing the image restoration unit 20j to execute an inference process. Upon receiving the trigger signal (Trigger 1) sent from cores N-2 and N-1, the image restoration unit 20j inputs a partial image corresponding to the repair code candidate area extracted by cores N-2 and N-1 into a neural network and executes an inference process to generate a repaired image by repairing the partial image (Image Restoration 1). The repaired image generated by Image Restoration 1 is sent to cores N-2 and N-1. Based on the repaired image generated by Image Restoration 1, cores N-2 and N-1 determine grid positions indicating the positions of each cell of the code and execute a decoding process for the repaired image based on the determined grid positions.

[0126] On the other hand, when core i+1 and core i+2 constituting the second decoding unit 203 extract a repair code candidate area, they output a trigger signal (trigger 2) to the image restoration unit 20j, causing the image restoration unit 20j to execute an inference process. Upon receiving the trigger signal (trigger 2) sent from core i+1 and core i+2, the image restoration unit 20j inputs the partial image corresponding to the repair code candidate area extracted by core i+1 and core i+2 into a neural network and executes an inference process to generate a repaired image by repairing the partial image (image restoration 2). The repaired image generated by image restoration 2 is sent to core i+1 and core i+2. Based on the repaired image generated by image restoration 2, core i+1 and core i+2 determine grid positions indicating the position of each cell of the code and execute a decoding process for the repaired image based on the determined grid positions.

[0127] Similarly, image restoration 3 is executed in response to a trigger signal (trigger 3) sent from core N-2 and core N-1, image restoration 4 is executed in response to a trigger signal (trigger 4) sent from core i+1 and core i+2, image restoration 5 is executed in response to a trigger signal (trigger 5) sent from core N-2 and core N-1, image restoration 6 is executed in response to a trigger signal (trigger 6) sent from core i+1 and core i+2, and image restoration 7 is executed in response to a trigger signal (trigger 7) sent from core N-2 and core N-1.

[0128] As shown in this time chart, the inference processing by the image restoration unit 20j and the first decoding processing by the first decoding unit 202 are executed in parallel. For example, while the first decoding unit 202 continues to perform code region extraction, grid positioning, and decoding processing, the inference processing by the image restoration unit 20j is executed once or multiple times. Also, while the first decoding unit 202 continues to perform code region extraction, grid positioning, and decoding processing, the second decoding unit 203 extracts a repair code region, outputs a trigger signal, positions a grid, and executes second decoding processing. In this way, the image restoration unit 20j only needs to specialize in repairing a partial image corresponding to a repair code candidate region, thereby increasing the inference processing speed.

[0129] At least a portion of the time during which the second decoding unit 20j is performing the second decoding process includes a pause period during which the image restoration unit 20j does not perform inference processing. That is, as shown in FIG. 18 , after the image restoration unit 20j completes image restoration 1, the second decoding unit 203 starts the second decoding process. However, a predetermined period is provided before the image restoration unit 20j starts the next image restoration 2. This predetermined period is a pause period during which the image restoration unit 20j does not perform image restoration. This pause period is a period during which the image restoration unit 20j does not perform inference processing. By providing this pause period, the load on the image restoration unit 20j can be reduced and heat generation in the AI ​​chip 23 can be suppressed. Note that the processing time for grid positioning may be long or short depending on the image to be restored. While FIG. 18 illustrates an example in which the processing of each core is performed alternately, depending on the processing time for grid positioning, the processing of each core does not necessarily occur alternately. That is, trigger issuance and grid positioning may be performed in order starting from the core with the least processing time.

[0130] As described above, the setting unit 20e is configured to allow the user to select either "applying an image restoration filter" or "not applying an image restoration filter." Applying an image restoration filter means performing image restoration, and not applying an image restoration filter means not performing image restoration. Therefore, the setting unit 20e also corresponds to a part that sets whether or not the second decoding unit 203 performs the second decoding process. Since this setting is performed by the user, the setting unit 20e is configured to be able to accept user operations. The setting is not limited to applying or not applying an image restoration filter, but may also be, for example, applying or not applying an inference process (restoration process), or applying or not applying an AI process. The user operation may, for example, be the operation of a button or the like displayed on a user interface screen.

[0131] The optical information reading device 1 is operated after the above-mentioned tuning process. For example, if the workpiece W is a label, the code is printed clearly, and if appropriate conditions are set through the tuning process, a high reading rate can be achieved. However, for workpieces W made of metal or resin, even if optimal conditions are set during the tuning process, reading may be difficult due to changes in the read image caused by weak ambient light, or the optical design and algorithm design may make it impossible to read due to extremely low contrast.

[0132] By performing the image restoration algorithm executed by the image restoration unit 20j in this example in parallel with the decoding process without image restoration, it is possible to improve the reading performance for workpieces W made of metal, resin, etc. while maintaining conventional reading performance, and to increase reading stability. In addition, by applying the image restoration algorithm, it becomes possible to read workpieces W that cannot be read using conventional optical designs and algorithms.

[0133] To give a specific example, in a process of reading a code printed on a sticker attached to a box in a factory, if there are no restrictions on the installation conditions of the optical information reading device 1 and optimal installation is possible, stable operation is possible even with conventional reading performance, so the image restoration filter can be disabled and takt time can be prioritized.

[0134] Also, in a certain factory, there is a process for reading codes that are clearly printed on metal, and although the optimal conditions are determined in the tuning process when setting up the reading, in actual operation there are cases where reading becomes unstable due to scratches on the metal surface, variations in the reading position and angle, changes in the brightness of the surrounding environment, etc. In such cases, applying an image restoration filter to improve the image quality makes it more resistant to variations in conditions such as those mentioned above, and improves reading performance.

[0135] In addition, some workpieces cannot be read using conventional algorithms, but in such cases, applying an image restoration filter can make them readable. For example, even if the code is severely damaged or the contrast is unclear, applying an image restoration filter can restore the contrast and scratches, making it possible to read the code.

[0136] The second decoding process executed when an image restoration filter is applied performs decoding on a restored image, and therefore often takes longer than the first decoding process, which performs decoding on a normal read image. Therefore, the first decoding unit 202 can execute the first decoding process in a shorter time than the time required for the second decoding process by the second decoding unit 203. In this example, based on this premise, the setting unit 20e can change the decoding timeout time, which is the time limit for the decoding process.

[0137] Specifically, when the second decoding process is set to be executed, the setting unit 20e is configured to be able to set a decoding timeout time longer than the time required to execute the second decoding process, and when the second decoding process is set not to be executed, the setting unit 20e is configured to be able to set a decoding timeout time shorter than the time required to execute the second decoding process. This allows a decoding timeout time longer than the time required to execute a typical second decoding process to be set during operation when the second decoding process is executed, thereby ensuring reliable decoding of the restored image. On the other hand, when the second decoding process is set not to be executed, only the first decoding process, which generally can be decoded in a shorter time than the second decoding process, is executed, so a short decoding timeout time corresponding to the first decoding process can be set. By changing the decoding timeout time in this manner, it is possible to set decoding timeout times corresponding to both the first decoding process and the second decoding process.

[0138] 18, the first decoding unit 202 and the second decoding unit 203 are shown as separate units, but as shown in FIG. 19, the first decoding unit 202 and the second decoding unit 203 may be combined into a decoding unit 20f. The decoding unit 20f performs a second decoding process on the restored image in parallel with a first decoding process on the scanned image generated by the camera 5. In this way, the parts that perform the decoding process can be arbitrarily divided or integrated on the hardware. The following description will be given using an example in which the parts that perform the decoding process are integrated into one unit, but this is not limiting and the same applies to cases in which the parts are divided into multiple units.

[0139] When the setting unit 20e sets the second decoding process not to be executed, the decoding unit 20f allocates processing resources used for the second decoding process to the first decoding process, thereby speeding up the first decoding process compared to when the second decoding process is set to be executed. Processing resources include, for example, memory usage areas and cores constituting a multi-core CPU. Specifically, when the setting unit 20e sets the second decoding process not to be executed, the number of cores responsible for the first decoding process is increased compared to when the setting unit 20e sets the second decoding process to be executed. Furthermore, when the setting unit 20e sets the second decoding process not to be executed, the memory area used for the first decoding process is expanded compared to when the setting unit 20e sets the second decoding process to be executed. Increasing the number of cores and expanding the memory area may be performed together, or either one of them may be performed alone. This maximizes the performance of the multi-core CPU to speed up the first decoding process.

[0140] Prior to image restoration, the decoding unit 20f extracts code candidate regions from the scanned image that are likely to contain a code. Specifically, when using a neural network to restore an image, if the image input to the neural network is large, the computation time increases. Furthermore, from the perspective of high-speed processing, it is desirable to minimize the number of decoding attempts. To address these issues, this example employs a heat map-based search method rather than a line search-based method to ensure reliable extraction of code candidate regions, even if it takes some time. For example, the decoding unit 20f quantifies the features of the code, generates a heat map in which the magnitude of the feature is assigned to each pixel value, and extracts code candidate regions from the heat map that are likely to contain a code. A specific example is a method of extracting feature portions (e.g., finder patterns) of a 2D code from areas that are relatively hot (large feature amounts) in the heat map. If multiple feature portions are acquired, they can be prioritized and extracted, and stored in the RAM 41, the storage unit 30, etc.

[0141] After the decoding unit 20f extracts the code candidate region, it outputs the partial image corresponding to the extracted code candidate region to the image restoration unit 20j. The image restoration unit 20j inputs the partial image corresponding to the code candidate region extracted by the decoding unit 20f into a neural network and executes an inference process to generate a restored image by restoring the partial image. This reduces the calculation time.

[0142] The decoding unit 20f may also be configured to extract a code candidate area in the scanned image that is large enough to contain the code to be read, as determined by the tuning process described above, and that is likely to contain the code. The extracted image is then reduced or enlarged to a predetermined size before being input to the neural network, and the restored image restored by the neural network is decoded. By enlarging or reducing the extracted image to a predetermined size and inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range, thereby stably achieving the image restoration effect of the neural network. Although the code size to be read may vary, the size of the code ultimately input to the neural network can be fixed, thereby maintaining a constant balance between high processing speed and ease of decoding.

[0143] The flowcharts of the first decoding process and the second decoding process will be described in detail below with reference to FIG. 20. In step SE1 after the start, the processor 20 causes the camera 5 to generate a read image and acquires the read image. The read image acquired in step SE1 is input to the decoding unit 20f. In step SE2, the first decoding process is executed on the read image acquired in step SE1 without performing image restoration. If the first decoding process is successful, the process proceeds to step SE4 to execute termination processing, i.e., output processing of the decoded result. If the first decoding process fails in step SE2, the process returns to step SE2 and executes the first decoding process again. When a predetermined decoding timeout period has elapsed, the first decoding process for the read image is stopped.

[0144] On the other hand, in the route proceeding from step SE1 to step SE5, the decoding unit 20f extracts a repair code area from the read image. Specifically, it extracts a code candidate area that is large enough to contain the code to be read, set by the tuning process, and that is likely to contain the code in the read image. At this time, if multiple repair code areas are extracted, they are temporarily saved as candidate areas R1, R2, .... An example of the extracted repair code area is shown in Figure 2. 1 Then, the process proceeds to step SE6, where the value of k is set to 1.

[0145] In step SE7, a resizing process is performed on the partial image corresponding to the candidate region Rk. The resizing process reduces or enlarges the partial image corresponding to the repair code region extracted in step SE5 to a predetermined size. This resizing process allows the size of the image to be input to the neural network, which will be described later, to be set to a predetermined size in advance. In the example shown in FIG. 21, the size of the partial image to be input to the neural network is 256 pixels x 256 pixels, but the size of the partial image is not limited to this and can be resized to any size taking into account processing speed, etc.

[0146] In step SE8, it is determined whether the black-and-white inversion setting is set to "both." In other words, image restoration is simply a mapping in mathematical terms, and it may be difficult to achieve mapping from white to black, black to white, black to black, and white to white using a single neural network. Therefore, black-and-white inversion processing is performed based on the properties so that the image is printed black on a white background. Depending on how the neural network is trained, the image may be printed white on a black background. An example of black-and-white inversion processing is shown in Figure 21. Details of black-and-white inversion processing will be described later.

[0147] If the determination in step SE8 is YES and the black-and-white inversion setting is "both," the process proceeds to step SE9, where the resized image is input to the image restoration unit 20j and image restoration is performed. Thereafter, the process proceeds to step SE10, where the decoding unit 20f performs a second decoding process on the restored image. If the second decoding process is successful in step SE10, the process proceeds to step SE4 and executes termination processing. If the second decoding process fails in step SE10, the process proceeds to step SE11, where the resized image is subjected to black-and-white inversion processing. Thereafter, in step SE11, the image after the black-and-white inversion processing is input to the image restoration unit 20j and image restoration is performed (see FIG. 21). Next, the process proceeds to step SE13, where the decoding unit 20f performs a second decoding process on the restored image. If the second decoding process is successful in step SE13, the process proceeds to step SE4 and executes termination processing. If the second decoding process fails in step SE13, the process proceeds to step SE14. In step SE14, it is determined whether or not there are any more candidate regions extracted in step SE5, and if there are no more candidate regions, the process proceeds to step SE4 to execute the termination process.

[0148] On the other hand, if step SE8 returns NO and the black-and-white inversion setting is not set to "both," the process proceeds to step SE15, where it is determined whether the black-and-white inversion setting is set to "OFF." If step SE15 returns NO and the black-and-white inversion setting is not set to "OFF," the process proceeds to step SE16, where the image after the resizing process is inverted. Thereafter, in step SE17, the image after the black-and-white inversion process is input to the image restoration unit 20j, where image restoration is performed. Next, the process proceeds to step SE18, where the decoding unit 20f performs a second decoding process on the restored image. If the second decoding process is successful in step SE18, the process proceeds to step SE4, where an end process is performed. If the second decoding process is unsuccessful in step SE18, the process proceeds to step SE14. In step SE14, it is determined whether or not there are still candidate areas. If there are no candidate areas, the process proceeds to step SE4, where an end process is performed.

[0149] If it is determined in step SE14 that there are still candidate areas, the process proceeds to step SE19, where 1 is added to the value of k, and the process proceeds to step SE7. In step SE7, the value of k has increased by 1 since the previous flow, so a similar resizing process is performed on the partial image corresponding to the next candidate area Rk. Steps from SE8 onwards are as described above. In other words, if decoding is successful in either the first decoding process or the second decoding process, the decoded result is output at that point, and decoding processes are not performed on other candidate areas; instead, a scanned image of the next work W is obtained, and similar processes are performed.

[0150] The post-processing unit 201 shown in FIG. 21 performs a process of synthesizing a partial image that has not been restored after resizing with a partial image that has been restored by the image restoration unit 20j after resizing. In other words, when image restoration using a neural network is performed, the code before restoration may be visually recognizable, but the image restoration may not be successful, or the cells may become round after image restoration. To address these issues, overlapping the partial image that has not been restored with the partial image that has been restored at an appropriate ratio may facilitate subsequent decoding. The overlap ratio between the partial image that has not been restored and the partial image that has been restored may be a predetermined fixed value or may be a non-fixed value. For example, it may be determined dynamically during scanning or by optimization through pre-evaluation (tuning) based on the stability of restoration by the image restoration unit 20j. For example, in pre-evaluation, the ratio may be adjustable by the user using a scale bar or the like, allowing the user to adjust the ratio while the combined image is presented to the user.

[0151] (Bank switching function) In this embodiment, the parameter set storage unit 30c is configured to store parameter sets, each of which includes a set of parameters constituting the imaging conditions of the camera 5 and parameters constituting the processing conditions in the decoding unit 20f. A parameter set can be called a bank. A plurality of banks are provided, each storing different parameters. For example, the parameter set storage unit 30c stores a plurality of imaging conditions and code conditions, including first imaging conditions and code conditions set by the tuning execution unit 20d and second imaging conditions and code conditions, as separate parameter sets.

[0152] The optical information reading device 1 is configured to be able to switch from one parameter set including first imaging conditions and code conditions to another parameter set including second imaging conditions and code conditions, or vice versa, among the multiple parameter sets stored in the parameter set storage unit 35c. The parameter set switching can be performed by the processor 20, a user, or a switching signal from an external control device such as the PLC 101. The user can switch parameter sets by, for example, operating a parameter set switching unit incorporated in the user interface. By enabling the parameter set switching unit, the parameter set in that bank is used during operation of the optical information reading device 1, and by disabling the parameter set switching unit, the parameter set in that bank is not used during operation of the optical information reading device 1. In other words, the parameter set switching unit is used to switch from one parameter set to another.

[0153] In this example, when the decoding unit 20f is operating using one parameter set and fails to perform the second decoding process on a restored image, it switches to another parameter set and performs image restoration and the second decoding process on a scanned image captured under different conditions. For example, when the decoding unit 20f detects that the second decoding process has failed, it switches to a parameter set other than the current parameter set and performs image restoration and the second decoding process. If it detects that the second decoding process has also failed with this parameter set, it switches to a parameter set that has not been used before and performs image restoration and the second decoding process. Switching parameter sets may change not only the imaging conditions of the camera 5 but also the code size to be read. In this case, the size used to extract code candidate areas where a code is likely to exist also changes. As a result, the reduction or enlargement ratio of the image input to the neural network also changes, changing the results of image restoration.

[0154] (Reads codes printed in black on a white background and in white on a black background) For example, when the optical reader 1 is used for the purpose of commodity traceability in a factory or warehouse, the color of the work W is not necessarily white, but may be black or a dark color close to black. If the color of the work W is black, the color of the code printed on it is white or a light color close to white. Also, depending on how the light hits it, the relationship between white and black may appear reversed. In other words, there are sites where codes printed in black on a white background and codes printed in white on a black background are mixed, and the optical reader 1 of this embodiment is configured to be able to handle both of these.

[0155] A specific explanation will be given below with reference to Figs. 22 and 23. Fig. 22 is a diagram illustrating a case where a first neural network trained with a code printed in black on a white background and a second neural network trained with a code printed in white on a black background are used to handle inversion of black and white in a code. Fig. 22A shows a method in which a scanned image (the image at the left end) is input to the first neural network and the second neural network, respectively, and decoded results are output from the first neural network and the second neural network; this method is called a parallel execution method. Because the scanned image is an image of a code printed in black on a white background, decoding is successful when the image is restored using the first neural network, but decoding fails when the image is restored using the second neural network.

[0156] In the example shown in FIG. 22A, the neural network storage unit 30d stores a first neural network and a second neural network. The decoding unit 20f inputs the scanned image to both the first neural network and the second neural network. The generated restored image is decoded and the successfully decoded image is output.

[0157] In FIG. 22B, the scanned image (the image on the left) is first input only to the second neural network, and an attempt is made to decode the restored image. Only if the decoding process fails is the scanned image input to the first neural network to generate a restored image. This method uses a different network after a failure.

[0158] In other words, in the example shown in FIG. 22B, the neural network memory unit 30d stores a first neural network and a second neural network, and the decoding unit 20f inputs the read image only to the second neural network, and then inputs the read image to the first neural network to generate a repaired image, and decodes the repaired image generated by the first neural network, only if decoding of the repaired image generated by inputting the read image to the second neural network fails.

[0159] FIG. 22C shows a method for selecting a neural network to input a scanned image based on a user setting. That is, the setting unit 20e shown in FIG. 6 is a part that accepts a selection of either the first neural network or the second neural network to input a scanned image to. For example, when setting up the optical information reading device 1, a user interface that allows the user to select either the "first neural network" or the "second neural network" can be generated and displayed on the display unit 6 or the display 100b of the setting device 100, thereby accepting the user's selection.

[0160] The selection method may be a method of directly selecting the type of neural network as described above, or a method of selecting the scanned image to be input. In the method of selecting the scanned image to be input, a user interface that allows the user to select either "an image of a scanned image of a code printed in black on a white background" or "an image of a scanned image of a code printed in white on a black background" can be generated and displayed on the display unit 6 or the display 100b of the setting device 100, and the user's selection can be accepted. Selection of "an image of a scanned image of a code printed in black on a white background" corresponds to selection of "first neural network," and selection of "an image of a scanned image of a code printed in white on a black background" corresponds to selection of "second neural network."

[0161] The result of the user's selection is received by the processor 20. The processor 20 is provided with a network selection module 20k that constitutes part of the decoding unit 20f. If the "first neural network" is selected, the network selection module 20k inputs the scanned image to the first neural network but not to the second neural network. On the other hand, if the "second neural network" is selected, the network selection module 20k inputs the scanned image to the second neural network but not to the first neural network.

[0162] The case shown on the left side of FIG. 22C is when the scanned image is set to be an image of a code printed in black on a white background. In this case, the network selection module 20k selects the first neural network. The decoding unit 20f then inputs the scanned image (the image at the left end) into the first neural network to generate a repaired image, and decodes the repaired image generated by the first neural network. On the other hand, the case shown on the right side of FIG. 22C is when the scanned image is set to be an image of a code printed in white on a black background. In this case, the network selection module 20k selects the second neural network. The decoding unit 20f then inputs the scanned image (the image at the left end) into the second neural network to decode the repaired image generated by the second neural network.

[0163] FIG. 23A illustrates a case where the neural network storage unit 30d stores either the first neural network or the second neural network, and only one of the neural networks is used to handle the black-and-white inversion of a code. FIG. 23A illustrates a case where the decoding unit 20f generates a black-and-white inverted image by performing black-and-white inversion on the scanned image (the image at the left edge). An image without black-and-white inversion and an image with black-and-white inversion are input to the first neural network. Because the scanned image is an image of a code printed in white on a black background, decoding fails when the image is restored using the first neural network. However, decoding is successful when the image after black-and-white inversion is restored using the first neural network to generate a restored image. That is, the decoding unit 20f performs a black-and-white inversion process to invert the black-and-white relationship of the read image so that the black-and-white relationship of the read image matches the black-and-white relationship that can be restored by the neural network stored in the neural network storage unit 30d, and then inputs the image to the neural network stored in the neural network storage unit 30d.

[0164] In the example shown at the top of FIG. 23B, the scanned image (the image on the left) is an image of a code printed in white on a black background, so when it is input to the first neural network, the decoding process fails. After that, when an image with black and white inversion is generated and input to the first neural network, the decoding process is successful.

[0165] In the example shown at the bottom of FIG. 23B, the scanned image (the image on the left) is an image of a code printed in black on a white background, so when it is input to the second neural network, the decoding process fails. After that, when an image with black and white inversion is generated and input to the second neural network, the decoding process is successful.

[0166] In other words, the decoding unit 20f is configured to input a read image that has undergone black-and-white inversion processing into the neural network stored in the neural network storage unit 30d only when it fails to decode a restored image generated by inputting a read image that has not undergone black-and-white inversion processing into the neural network stored in the neural network storage unit 30d.

[0167] FIG. 23C shows a method for determining whether to perform black-and-white inversion processing or not based on a setting made by a user. That is, the setting unit 20e shown in FIG. 6 is a part that accepts a selection of whether to perform black-and-white inversion processing or not. For example, when setting up the optical information reading device 1, a user interface that allows the user to select either "perform black-and-white inversion processing" or "not perform black-and-white inversion processing" can be generated and displayed on the display unit 6 or the display 100b of the setting device 100, so as to accept the user's selection.

[0168] The selection method may be a method of directly selecting whether or not to perform black-and-white inversion processing as described above, or a method of selecting a scanned image to be input. In the method of selecting a scanned image to be input, a user interface that allows the user to select either "a scanned image of a code printed in black on a white background" or "a scanned image of a code printed in white on a black background" can be generated and displayed on the display unit 6 or the display 100b of the setting device 100, and the user's selection can be accepted. When only the first neural network is stored and "a scanned image of a code printed in black on a white background" is selected, "black-and-white inversion processing not performed" is selected, and when only the first neural network is stored and "a scanned image of a code printed in white on a black background" is selected, "black-and-white inversion processing performed" is selected. Furthermore, when only the second neural network is stored, if "an image in which the scanned image is a code printed in black on a white background" is selected, "black and white inversion processing is performed," and when only the second neural network is stored, if "an image in which the scanned image is a code printed in white on a black background" is selected, "black and white inversion processing is not performed."

[0169] The result of the user's selection is received by the processor 20. The processor 20 includes a black-and-white inversion processing module 20l that forms part of the decoding unit 20f. If "perform black-and-white inversion processing" is selected, the black-and-white inversion processing module 20l performs black-and-white inversion processing on the read image, but if "do not perform black-and-white inversion processing" is selected, the black-and-white inversion processing module 20l does not perform black-and-white inversion processing on the read image.

[0170] The case shown in the upper part of FIG. 23C is a case where only the first neural network is stored and the scanned image is set to be an image of a code printed in black on a white background. In this case, the black-and-white reversal processing module 201 inputs the image directly to the first neural network without performing black-and-white reversal processing. On the other hand, the case shown in the lower part of FIG. 23C is a case where only the first neural network is stored and the scanned image is set to be an image of a code printed in white on a black background. In this case, the black-and-white reversal processing module performs black-and-white reversal processing. The image after black-and-white reversal processing is then input to the first neural network.

[0171] (Relationship between contrast and matching level) The matching level is used, for example, as a score during tuning, or to manage print quality / reading performance during operation. For example, the matching level is defined as a positive integer value between 0 and 100, and a level of 50 or higher can be designed to achieve a 100% read rate when a read rate test is conducted. Also, a matching level of 0 means that the code cannot be read, and a matching level of 1 is the lowest value at which the code can be read.

[0172] 24 is a graph showing the relationship between contrast and matching level, where the solid line shows the relationship between the contrast and matching level of an image (restored image) restored by the image restoration unit 20j, and the dashed line shows the relationship between the contrast and matching level of an image (read image) that has not been restored by the image restoration unit 20j. As is clear from this graph, the matching level of the restored image is generally higher than that of the read image, but when the image quality deteriorates beyond a certain level, the matching level drops sharply.

[0173] (Matching level calculation process) A matching level is set for each algorithm, and typically one algorithm is used for each code type. However, in this embodiment, an image restoration function is implemented, and two algorithms (one with image restoration and one without image restoration) are executed for each code type. The result of the algorithm that first successfully decodes the image is output. Therefore, if the two algorithms alternately succeed in decoding, the matching level value may become unstable. Furthermore, since image restoration makes the image easier to read, conventional methods of calculating the matching level tend to result in a higher matching level than existing methods. In response to these issues, this embodiment calculates a weighted sum of the matching levels of the two algorithms and adjusts the matching level value. A specific example will be described below based on the flowchart shown in FIG. 25.

[0174] In step SF1 after starting, the read image is input to the processor 20. In step SF2, the first decoding unit 202 executes the first decoding process without image restoration. In step SF3, the matching level A (MLA) by the first decoding process is calculated. Meanwhile, in step SF4, the read image is input to the image restoration unit 20j and image restoration process is executed. Thereafter, the process proceeds to step SF5, where the second decoding unit 203 executes the second decoding process on the restored image. In step SF6, the matching level B (MLB) by the second decoding process is calculated. Note that if decoding is not successful in step SF2, MLA = 0, and if decoding is not successful in step SF5, MLB = 0.

[0175] In step SF7, the MLA and MLB values ​​are adjusted using, for example, an averaging function, etc. Furthermore, regardless of whether decoding is successful in step SF2 or SF5, a process of weighting the MLA value is performed.

[0176] (Example of user interface) FIG. 26 is a diagram showing an example of a user interface 300 displayed on the display unit 101 of the setting device 100 during operation. The setting device 100 generates the user interface 300 and displays it on the display unit 101. The user interface 300 is provided with a first image display area 301 in which an image currently captured by the camera 5 (a live view image) is displayed, a second image display area 302 in which a read image is displayed, and a third display area 303 in which a restored image restored by the image restoration unit 20j is displayed. This allows the read image and the restored image to be presented to the user in a form that allows them to compare them. In addition, the user can understand what the image to be decoded or what the image after the decoding process is performed looks like.

[0177] (Example of an image sensor with an AI chip) FIG. 27 shows an example in which an imaging element 5a with an AI chip is used, and the imaging element 5a of the camera 5 can be configured as shown in each figure. FIG. 27A shows a packaged imaging element 5a equipped with an AI chip that performs image restoration. After the scanned image is restored, it is output to the processor 20. FIG. 27B shows a packaged imaging element 5a equipped with an AI chip that extracts a repair code area and performs image restoration. After extracting a repair code area from the scanned image, it repairs a partial image corresponding to that area and outputs it to the processor 20. FIG. 27C shows a packaged imaging element 5a equipped with an AI chip that extracts a repair code area. After extracting a repair code area from the scanned image, it outputs a partial image corresponding to that area to the processor 20.

[0178] In the example shown in FIG. 27A, the scanned image acquired by the image sensor 5a may be output to the processor 20, and the image restoration may be performed by an AI chip, and the restored image may be output to the processor 20. The same applies to the examples shown in FIGS. 27B and 27C.

[0179] In this embodiment, an optical information reading device is disclosed that can repair both codes printed in black on a white background and code images printed in white on a black background by storing either a first neural network for repairing codes printed in black on a white background or a second neural network for repairing codes printed in white on a black background and inputting code images into either or both of the neural networks. However, it is also possible to repair both codes printed in black on a white background and codes printed in white on a black background using a single neural network. Hereinafter, the neural network having the repair function for code images printed in black on a white background and code images printed in white on a black background will be referred to as the third neural network.

[0180] The third neural network is generated as follows. A pair of input image and correct answer image is required for neural network training. When training a code image whose original image is black printed on a white background, the third neural network is trained using a code image whose original image is black printed on a white background and a correct answer image whose original image is black printed on a white background. On the other hand, when training a code image whose original image is white printed on a black background, the third neural network is trained using a code image whose original image is white printed on a black background and a correct answer image whose original image is white printed on a black background. This makes it possible to generate a single neural network that has the ability to repair both code images whose original image is black printed on a white background and code images whose original image is white printed on a black background.

[0181] By learning in the above manner, the third neural network can automatically determine whether the code image is printed in black on a white background or in white on a black background, and restore the image quality. Therefore, it is possible to properly restore code images printed in black on a white background and in white on a black background without preparing multiple neural networks or performing black-and-white inversion.

[0182] (Effects of the embodiment) As described above, according to this embodiment, when a first neural network trained on a code with black printed on a white background is stored in the neural network storage unit 30d, a restored image is generated by directly inputting a scanned image of a code with black cells on a white background into the first neural network, thereby improving the decoding success rate. On the other hand, when the scanned image is a code with white cells on a black background, the black and white relationship of the scanned image is inverted to create a black and white relationship that can be restored by the first neural network. Therefore, a restored image is generated by the first neural network, improving the decoding success rate.

[0183] Furthermore, if the second neural network has been trained using a code with white cells printed on a black background, a restored image can be generated by directly inputting a scanned image of a code with white cells on a black background into the second neural network. On the other hand, if the scanned image is a code with black cells on a white background, the black-and-white relationship of the scanned image can be reversed to create a black-and-white relationship that can be restored by the second neural network. Thus, a restored image can be generated by the second neural network. In other words, whether the code is one with black cells on a white background or one with white cells on a black background, a restored image can be generated by the first neural network or the second neural network, thereby improving the decoding success rate.

[0184] In addition, a decodable repaired image can be generated by inputting a scanned image of a code with black cells on a white background into the first neural network, and a decodable repaired image can be generated by inputting a scanned image of a code with white cells on a black background into the second neural network. Therefore, even if a code with black cells on a white background and a code with white cells on a black background are mixed, the decoding success rate is improved.

[0185] The above-described embodiments are merely examples in all respects and should not be construed as limiting. Furthermore, all modifications and variations within the scope of the claims are within the scope of the present invention. [Industrial Applicability]

[0186] As described above, the optical information reading device according to the present invention can be used, for example, to read a code attached to a workpiece. [Explanation of symbols]

[0187] 1 Optical information reader 4. Lighting section 5. Camera 20f Decoder 20k Network Selection Module 20l Black and white inversion processing module 30d Neural network memory section

Claims

1. An optical information reading device that reads a code attached to a workpiece, a camera that takes an image of a workpiece to which a code has been added and generates a scanned image including the code; a memory unit that stores the structure and parameters of a neural network for estimating an ideal image corresponding to the read image generated by the camera, the ideal image being generated in advance by machine learning a plurality of defective images having portions inappropriate for reading a code and a plurality of ideal images corresponding to the plurality of defective images; a decoding unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit, executes an inference process to generate a restored image by restoring the scanned image, and decodes the restored image, the storage unit stores either a first neural network that performs machine learning on the defective image and the ideal image, which include a code having black cells on a white background, so that the repair effect on the code having black cells on a white background is higher than the repair effect on the code having white cells on a black background, or a second neural network that performs machine learning on the defective image and the ideal image, which include a code having white cells on a black background, so that the repair effect on the code having white cells on a black background is higher than the repair effect on the code having black cells on a white background, The decoding unit performs a black-and-white inversion process to invert the black-and-white relationship of the read image acquired by the camera so that the black-and-white relationship of the read image matches the black-and-white relationship that can be restored by the neural network stored in the memory unit, and then inputs the result into the neural network stored in the memory unit.

2. An optical information reading device that reads a code attached to a workpiece, a camera that takes an image of a workpiece to which a code has been added and generates a scanned image including the code; a memory unit that stores the structure and parameters of a neural network for estimating an ideal image corresponding to the read image generated by the camera, the ideal image being generated in advance by machine learning a plurality of defective images having portions inappropriate for reading a code and a plurality of ideal images corresponding to the plurality of defective images; a decoding unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit, executes an inference process to generate a restored image by restoring the scanned image, and decodes the restored image, the storage unit stores a first neural network that performs machine learning on the defective image and the ideal image, which include a code having black cells on a white background, to thereby achieve a higher repair effect on the code having black cells on a white background than on a code having white cells on a black background; and a second neural network that performs machine learning on the defective image and the ideal image, which include a code having white cells on a black background, to thereby achieve a higher repair effect on the code having white cells on a black background than on a code having black cells on a white background. The decoding unit inputs the read image to at least one of the first neural network and the second neural network.

3. 3. The optical information reading device according to claim 2, The decoding unit inputs the read image to both the first neural network and the second neural network, and decodes the generated restored image.

4. 3. The optical information reading device according to claim 2, The decoding unit of the optical information reading device inputs the read image into one of the first and second neural networks to generate a repaired image only if it fails to decode the repaired image generated by inputting the read image into the other neural network, and decodes the repaired image generated by the other neural network.

5. 3. The optical information reading device according to claim 2, a setting unit that accepts a selection of either the first neural network or the second neural network to which the scanned image is to be input, The decoding unit inputs the read image into the neural network accepted by the setting unit to generate a restored image, and decodes the restored image generated by the neural network.

6. 2. The optical information reading device according to claim 1, The decoding unit inputs the read image on which the black-and-white inversion process has been performed and the read image on which the black-and-white inversion process has not been performed into a neural network stored in the memory unit.

7. 2. The optical information reading device according to claim 1, The decoding unit inputs the read image that has undergone the black-and-white inversion processing into the neural network stored in the memory unit only when it fails to decode the restored image generated by inputting the read image that has not undergone the black-and-white inversion processing into the neural network stored in the memory unit.

8. 2. The optical information reading device according to claim 1, a setting unit that accepts a selection as to whether or not the black-and-white reversal process is to be performed, An optical information reading device in which, when the execution of the black-and-white inversion process is selected in the setting unit, the decoding unit performs the black-and-white inversion process on the read image and then inputs the result into a neural network stored in the memory unit.

9. 9. The optical information reading device according to claim 1, a lighting unit having a plurality of groups of light-emitting elements for illuminating the code; a light-transmitting plate provided in front of one group of the lighting units; and a polarizing plate provided in front of the other group of illumination units.

10. 9. The optical information reading device according to claim 1, a lighting unit having a plurality of groups of light-emitting elements for illuminating the code; a light-transmitting plate provided in front of one group of the lighting units; and a diffusion plate provided in front of the other group of illumination units.

Images