Optical information reader

Abstract

To lower the lower limit of PCC capable of performing decoding processing by automatically applying super-resolution processing by a neural network when the PCC is small to the extent that decoding processing cannot be performed.SOLUTION: An optical information reader includes: a storage part for storing neural network structure and parameters for outputting an enlarged image obtained by enlarging PPC for showing the number of pixels of each module for composing a code included in a read image generated by a camera; and a processor for inputting a read image generated by the camera to the neural network composed of structure and parameters stored in the storage part, generating an enlarge image obtained by enlarging a PPC value of the inputted read image, and executing decoding processing to the enlarged image.SELECTED DRAWING: Figure 19

Description

Technical Field

[0001] The present invention relates to an optical information reading device for optically reading information.

Background Art

[0002] In recent years, so-called traceability, which enables, for example, tracking the distribution route of an article from the manufacturing stage to the consumption stage or the disposal stage, has been regarded as important, and code readers for this purpose have become widespread. In addition to traceability, code readers are also used in various fields.

[0003] Generally, a code reader is configured to be able to photograph a code such as a barcode or a two-dimensional code attached to a workpiece with a camera, cut out the code included in the obtained image by image processing, binarize it, and perform decoding processing to read information. Since it is a device for optically reading information, it is also called an optical information reading device.

[0004] As this type of optical information reading device, for example, as disclosed in Patent Document 1, there is known one provided with a machine learning device that learns a model structure showing the relationship between an image of a code acquired by a visual sensor of a robot and an image of an ideal code. Patent Document 1 describes that an image suitable for reading can be restored by applying the learning result by the machine learning device to the image of the code acquired by the visual sensor during operation.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Incidentally, in a read image obtained by photographing a code with a camera, there is a limit to the decoding process depending on the number of pixels per module (PPC) that makes up the code. In order to stably succeed in the decoding process, it is required that the PPC be a certain value or more, and if the PPC falls below a certain value, the decoding process may not be possible at all.

[0007] Here, various super-resolution processes are known as techniques for increasing the PPC, that is, the resolution of the read image. As an application example to an optical information reading device, for example, there is a super-resolution technique using a plurality of frame images. However, the super-resolution process using a plurality of frame images has problems such as the need to capture a plurality of frames, which takes time for the process, and difficulty in aligning the positions between the frame images.

[0008] The present invention has been made in view of such points, and an object thereof is to automatically apply super-resolution processing by a neural network when the PPC is small enough that decoding processing becomes impossible, thereby lowering the lower limit of the PPC at which decoding processing is possible.

Means for Solving the Problem

[0009] To achieve the above object, a first aspect of the present disclosure can be premised on an optical information reading device that reads a code attached to a workpiece. The optical information reading device includes a camera that photographs the code and generates a read image, a storage unit that stores the structure and parameters of a neural network that outputs an enlarged image obtained by enlarging the PPC indicating the number of pixels of each module that makes up the code included in the read image generated by the camera, and a processor. The processor inputs the read image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit, generates an enlarged image obtained by enlarging the PPC value of the input read image, and is configured to execute a decoding process on the enlarged image.

[0010] That is, for example, the farther the camera is from the workpiece, the smaller the code is photographed, so that the PPC becomes below the reading limit. Also, in the optical system of a handheld optical information reading device, even if the distance is not large, the module may be small and the PPC may be below the reading limit. When such a read image is input to a neural network, the processor generates an enlarged image in which the PPC value of the input read image is enlarged. As a result, a decoding process can be executed on the enlarged image having a PPC value larger than the reading limit.

[0011] In a second aspect of the present disclosure, the optical information reading device includes a PPC value acquisition unit that acquires the PPC value of a code included in a read image generated by the camera. The processor can generate the enlarged image when the PPC value acquired by the PPC value acquisition unit is equal to or less than a predetermined value, while not generating the enlarged image when the PPC value acquired by the PPC value acquisition unit exceeds the predetermined value.

[0012] According to this configuration, when the PPC value is equal to or less than a predetermined value, for example, below the reading limit, a decoding process can be enabled by generating an enlarged image. On the other hand, when the PPC value exceeds the reading limit, the decoding process is possible without generating an enlarged image. Therefore, in this case, the decoding result can be output early without performing the process by the neural network.

[0013] In a third aspect of the present disclosure, the PPC value acquisition unit can acquire a read image generated by the camera when the optical information reading device is set, and acquire the PPC value of a code included in the acquired read image.

[0014] According to this configuration, it is possible to acquire a read image generated by the camera at the time of setting the imaging conditions or the like of the optical information reading device, and acquire the PPC value of a code included in the acquired read image. Therefore, it is possible to determine at the time of setting whether it is necessary to generate an enlarged image, and automatically generate an enlarged image during operation if necessary.

[0015] In a fourth aspect of the present disclosure, when the processor fails in decoding the code included in the read image generated by the camera at the time of setting the optical information reading device, the enlarged image can be generated, and the decoding process can be executed on the enlarged image.

[0016] That is, in order to set imaging conditions and the like of the optical information reading device, it is necessary to succeed in the decoding process. If the PPC of the code of the read image used at the time of setting is below the reading limit, the setting may not be possible. In this case, when the processor generates an enlarged image, the decoding process becomes possible, and as a result, various settings can also be performed.

[0017] In a fifth aspect of the present disclosure, the processor may have a decoding processing unit that executes a first decoding process for decoding a read image generated by a camera and a second decoding process for decoding the enlarged image.

[0018] According to this configuration, for example, when the decoding process of the read image fails in the first decoding process, the decoding process of the enlarged image can be executed in the second decoding process. The first decoding process and the second decoding process may be executed in parallel, or one may be executed first and then the other.

[0019] In a sixth aspect of the present disclosure, the processor may have a core that executes the first decoding process and the second decoding process in parallel in different threads.

[0020] In a seventh aspect of the present disclosure, the decoding processing unit may be configured as a multi-core that can execute the first decoding process and the second decoding process on different cores. According to this configuration, since the core that executes the first decoding process and the core that executes the second decoding process are different cores, the first decoding process and the second decoding process can be executed in parallel, and the processing time is shortened.

[0021] In an eighth aspect of the present disclosure, the optical information reading device includes a region extraction unit that extracts a code candidate region where a code is likely to exist from the captured image generated by the camera. The processor can input a partial image corresponding to the code candidate region extracted by the region extraction unit to the neural network to generate the enlarged image.

[0022] According to this configuration, when the optical information reading device is in operation, when a code candidate region is extracted from the captured image generated by the camera, a partial image corresponding to the code candidate region is input to the neural network to generate an enlarged image. In this case, since there are almost no cases where a code exists in the entire captured image, the size of the partial image corresponding to the code candidate region is smaller than the size of the captured image. As a result, the size of the image input to the neural network becomes smaller, so the computational load on the processor is reduced, and as a result, the processing speed is increased.

[0023] In a ninth aspect of the present disclosure, the region extraction unit can search for a code in the captured image generated by the camera based on information for specifying the code, and extract a region including the searched code as the code candidate region.

[0024] According to this configuration, the optical information reading device can automatically extract the code candidate region without the user performing an operation for extracting the code candidate region, so the burden on the user can be reduced.

[0025] In a tenth aspect of the present disclosure, the optical information reading device includes a light irradiation unit that irradiates visible light of a color different from the ambient light toward the inside of the shooting field of view of the camera to form a mark. The region extraction unit can specify a portion corresponding to the mark formed by the light irradiation unit from the captured image generated by the camera, and extract the specified portion as the code candidate region.

[0026] According to this configuration, a mark is formed by visible light within the imaging field of view of the camera. For example, in an optical information reading device equipped with a handheld housing having a gripping portion for the user to grip during operation, by the user performing an operation of aligning the mark with the code, the range where the code is surely included can be imaged by the camera. Therefore, in the read image generated by the camera, it is highly likely that the code exists in the portion corresponding to the mark. By extracting the portion corresponding to this mark as a code candidate region, the possibility that the extracted region contains the code is further increased, and thus the success rate of the decoding process can be enhanced.

[0027] In an eleventh aspect of the present disclosure, the optical information reading device may further include a filter processing unit that executes a noise removal filter on the read image before inputting the read image generated by the camera to the neural network.

[0028] According to this configuration, factors that have an adverse effect during the generation of the enlarged image by the neural network can be eliminated.

Advantages of the Invention

[0029] As described above, by using a neural network that outputs an enlarged image in which the PPC indicating the number of pixels of each module constituting the code included in the read image is enlarged, the lower limit of the readable PCC can be lowered. Therefore, for example, decoding processing of a code photographed at a long distance or a code with a small module size becomes possible.

Brief Description of the Drawings

[0030]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Embodiments for Carrying Out the Invention

[0031] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following description of the preferred embodiments is merely illustrative in nature and is not intended to limit the present invention, its applications, or its uses.

[0032] (Stationary optical information reading device) FIG. 1 is a diagram schematically showing the operation of a stationary optical information reading device 1 according to an embodiment of the present invention. In this example, a plurality of workpieces W are placed on the upper surface of a conveyor belt B and conveyed in the direction of arrow Y in FIG. 1, and an optical information reading device 1 according to the embodiment is installed at a position above and separated from the workpiece W. The optical information reading device 1 is a code reader configured to photograph a code attached to the workpiece W and decode the code included in the photographed image to read information. In the example shown in FIG. 1, the optical information reading device 1 is a stationary type. When operating the stationary optical information reading device 1, it is fixed to a bracket or the like (not shown) so that the optical information reading device 1 does not move and is used. Note that the stationary optical information reading device 1 may be used while being gripped by a robot (not shown). Further, the code of the stationary workpiece W may be read by the optical information reading device 1. The operation time of the stationary optical information reading device 1 is when an operation of sequentially reading the codes of the workpieces W conveyed by the conveyor belt B is being performed.

[0033] In addition, a code is attached to the outer surface of each workpiece W. The code includes both a bar code and a two-dimensional code. Examples of the two-dimensional code include a QR code (registered trademark), a micro QR code, a data matrix (Data matrix; Data code), a Veri code, an Aztec code, a PDF417, a Maxi code, and the like. The two-dimensional code has a stacked type and a matrix type, but the present invention can be applied to any two-dimensional code. The code may be attached by directly printing or engraving on the workpiece W, or may be attached by printing on a label and then pasting it on the workpiece W, and the means and method are not limited.

[0034] The optical information reading device 1 is wired-connected to a computer 100 and a programmable logic controller (PLC) 101 via signal lines 100a and 101a respectively. However, it is not limited to this. The optical information reading device 1, the computer 100, and the PLC 101 may incorporate communication modules to wirelessly connect the optical information reading device 1 to the computer 100 and the PLC 101. The PLC 101 is a control device for sequence control of the conveyor belt B and the optical information reading device 1, and a general-purpose PLC can be used. The computer 100 can use a general-purpose or dedicated electronic computer, a portable terminal, or the like.

[0035] Also, during operation of the optical information reading device 1, it receives a reading start trigger signal that defines the start timing of code reading from the PLC 101 via the signal line 101a. Then, the optical information reading device 1 performs imaging and decoding of the code based on this reading start trigger signal. After that, the decoded result is transmitted to the PLC 101 via the signal line 101a. In this way, during operation of the optical information reading device 1, the input of the reading start trigger signal and the output of the decoded result are repeatedly performed between the optical information reading device 1 and an external control device such as the PLC 101 via the signal line 101a. Note that the input of the reading start trigger signal and the output of the decoded result may be performed via the signal line 101a between the optical information reading device 1 and the PLC 101 as described above, or may be performed via other signal lines not shown in the figure. For example, a sensor for detecting the arrival of the workpiece W may be directly connected to the optical information reading device 1, and the reading start trigger signal may be input from the sensor to the optical information reading device 1.

[0036] As shown in FIG. 2, the optical information reading apparatus 1 is provided with a box-shaped housing 2, a polarizing filter attachment 3, an illumination unit 4, a camera 5, a display unit 6, a power connector 7, and a signal line connector 8. Further, an indicator 9, an aiming light irradiation unit (light irradiation unit) 10, and operation buttons 11 and 12 are provided on the housing 2, and the indicator 9, the aiming light irradiation unit 10, and the operation buttons 11 and 12 are also components of the optical information reading apparatus 1.

[0037] The housing 2 has a shape that is long in a predetermined direction, but the shape of the housing 2 is not limited to the illustrated shape. The polarizing filter attachment 3 is detachably attached to the outer surface on the front side of the housing 2. Inside the housing 2, an illumination unit 4, a camera 5, an aiming light irradiation unit 10, a processor 20, a storage device 30, a ROM 40, a RAM 41, etc. are accommodated. The processor 20, the storage device 30, the ROM 40, and the RAM 41 are also components of the optical information reading apparatus 1.

[0038] The illumination unit 4 is provided on the front side of the housing 2. The illumination unit 4 is a part for illuminating at least the code of the work W by irradiating light forward of the optical information reading apparatus 1. As also shown in FIG. 3, the illumination unit 4 includes a first illumination unit 4a composed of a plurality of light emitting diodes (LEDs: Light Emission Diode), a second illumination unit 4b composed of a plurality of light emitting diodes, and an illumination driving unit 4c composed of an LED driver or the like for driving the first illumination unit 4a and the second illumination unit 4b. The first illumination unit 4a and the second illumination unit 4b are individually driven by the illumination driving unit 4c and can be separately turned on and off. The illumination driving unit 4c is connected to the processor 20, and the illumination driving unit 4c is controlled by the processor 20. Note that one of the first illumination unit 4a and the second illumination unit 4b may be omitted.

[0039] As shown in FIG. 2, a camera 5 is provided at the central part on the front side of the housing 2. The optical axis direction of the camera 5 substantially coincides with the light irradiation direction by the illumination unit 4. The camera 5 is a part that photographs a code and generates a read image. As shown in FIG. 3, the camera 5 includes an image sensor 5a that is attached to the workpiece W and receives reflected light from the code illuminated by the illumination unit 4, an optical system 5b having a lens and the like, and an AF module (auto focus module) 5c. The optical system 5b is configured such that light reflected from the portion of the workpiece W to which the code is attached is incident thereon, and the incident light is emitted toward the image sensor 5a and forms an image on the imaging surface of the image sensor 5a.

[0040] The image sensor 5a is an image sensor composed of a light receiving element such as a CCD (charge-coupled device) or a CMOS (complementary metal oxide semiconductor) that converts the image of the code obtained through the optical system 5b into an electrical signal. The image sensor 5a is connected to the processor 20, and the electrical signal converted by the image sensor 5a is input to the processor 20 as data of the read image. Further, the AF module 5c is a mechanism that performs focusing by changing the position and refractive index of the focusing lens among the lenses constituting the optical system 5b. The AF module 5c is also connected to the processor 20 and is controlled by the processor 20.

[0041] As shown in FIG. 2, a display unit 6 is provided on the side surface of the housing 2. The display unit 6 is composed of, for example, an organic EL display, a liquid crystal display, or the like. The display unit 6 is connected to the processor 20 and can display, for example, a code imaged by the imaging unit 5, a character string that is a decoding result of the code, a reading success rate, a matching level, and the like. The reading success rate is the average reading success rate when a plurality of reading processes are executed. The matching level is a reading margin indicating the ease of reading of a code for which decoding has been successful. This can be obtained from the number of error corrections generated during decoding and can be represented, for example, by a numerical value. The fewer the error corrections, the higher the matching level (reading margin), while the more the error corrections, the lower the matching level (reading margin).

[0042] A power cable (not shown) for supplying power from the outside to the optical information reading device 1 is connected to the power connector 7. Further, signal lines 100a, 101a, etc. for communicating with the computer 100 and the PLC 101 are connected to the signal line connector 8. The signal line connector 8 can be composed of, for example, an Ethernet connector, a serial communication connector such as RS232C, a USB connector, or the like.

[0043] An indicator 9 is provided on the housing 2. The indicator 9 is connected to the processor 20 and can be composed of a light emitter such as a light emitting diode. The operating state of the optical information reading device 1 can be notified to the outside by the lighting state of the indicator 9.

[0044] On the front side of the housing 2, a pair of aiming light irradiation units 10 are provided so as to sandwich the camera 5. As shown in FIG. 3, the aiming light irradiation unit 10 includes an aiming unit 10a composed of a light-emitting diode or the like, an aiming drive unit 10b that drives the aiming unit 10a, and an aiming lens 10c into which the light irradiated from the aiming unit 10a is incident. The aiming unit 10a irradiates light (aiming light) forward of the optical information reading device 1 to indicate the shooting range, the center of the field of view of the camera 5, a reference for the optical axis of the illumination unit 4, etc., and is provided with an element that irradiates light. Specifically, the aiming unit 10a irradiates visible light of a color different from the ambient light (for example, red, green, etc.) into the shooting field of view range of the camera 5, and forms a mark visible to the naked eye on the surface irradiated with the visible light. The mark may be various figures, symbols, characters, etc. The user can also install the optical information reading device 1 with reference to the light irradiated from the aiming unit 10a.

[0045] As shown in FIG. 2, on the side surface of the housing 2, operation buttons 11 and 12 used when setting the optical information reading device 1 or the like are provided. The operation buttons 11 and 12 include, for example, a select button, an enter button, etc. In addition to the operation buttons 11 and 12, for example, touch panel type operation means may be provided. The operation buttons 11 and 12 are connected to the processor 20, and the processor 20 can detect the operation states of the operation buttons 11 and 12. By operating the operation buttons 11 and 12, one can be selected from a plurality of options displayed on the display unit 6, or the selected result can be confirmed.

[0046] (Portable optical information reading device) In the above-described example, the case where the optical information reading device 1 is a stationary type is shown, but the present invention is applicable not only to the stationary optical information reading device 1. FIG. 4 shows a portable optical information reading device 1A, and the present invention can also be applied to the portable optical information reading device 1A as shown in this figure.

[0047] The housing 2A of the handheld optical information reading device 1A is long in the vertical direction. Note that the orientation of the optical information reading device 1A during use is not limited to the illustrated orientation and can be used in various orientations. For the sake of convenience in explanation, the vertical direction of the optical information reading device 1A is specified.

[0048] A display unit 6A is provided in the upper portion of the housing 2A. The display unit 6A is configured in the same manner as the display unit 6 of the stationary optical information reading device 1. The lower portion of the housing 2A is a grip portion 2B for the user to hold during operation. The grip portion 2B is a portion that can be held by a general adult with one hand, and its shape and size can be freely set. By holding this grip portion 2B, the optical information reading device 1A can be carried and moved. That is, this optical information reading device 1A is a portable terminal device and can also be called, for example, a handy terminal.

[0049] Similar to the stationary optical information reading device 1, the handheld housing 2A also houses an illumination unit, a camera, an aiming light irradiation unit, a processor, a storage unit, a ROM, a RAM, etc. (not shown). The optical axis of the illumination unit, the optical axis of the camera, and the optical axis of the aiming light irradiation unit are directed obliquely upward from near the upper end of the housing 2A. Also, a buzzer (not shown) is provided in the handheld housing 2A.

[0050] A plurality of operation buttons 11A and a trigger key 11B are provided on the grip portion 2B and its vicinity. The operation buttons 11A are the same as the operation buttons 11 of the stationary optical information reading device 1. When the user presses the trigger key 11B with the tip (upper end) of the optical reading device 1A facing the workpiece W, aiming light is irradiated from the tip of the optical reading device 1A, and a mark visible to the naked eye is formed on the surface irradiated with the aiming light. The user adjusts the orientation of the optical reading device 1A while visually observing the aiming light (mark) reflected on the surface of the workpiece W. If the aiming light is aligned with the code to be read, the code reading and decoding processes are automatically performed. When the reading is completed, a completion notification sound is emitted from the buzzer.

[0051] As an example of the use of the handheld optical information reading device 1A, there is an example of using the handheld optical information reading device 1A in a picking operation in a logistics warehouse. For example, when shipping an ordered product from a logistics warehouse, the necessary product is picked from the product shelves in the product warehouse. This picking operation is performed by a user with an order slip with a code moving to the product shelf and then collating the code on the order slip with the code attached to the product or the product shelf. In this case, the code on the order slip and the code attached to the product or the product shelf are alternately read by the handheld optical information reading device 1A.

[0052] FIG. 5 is a diagram simply showing the relationship between the optical axis of the aimer 10 of the handheld optical information reading device 1A and the optical axis of the camera 5. As shown in this figure, since the optical axis of the aimer 10 and the optical axis of the camera 5 are different, the center of the aimer light serving as a mark may be greatly deviated from the center of the field of view of the camera 5. For example, as shown as "short distance", "medium distance" and "long distance" in FIG. 6, depending on the distance between the handheld optical information reading device 1A and the work W, the relative position between the center of the aimer light 10d and the center of the field of view 5d of the camera 5 changes. In FIG. 6, the case where the aimer light 10d is in a + shape is shown, but it is not limited to this.

[0053] As shown in FIG. 4, in the handheld optical information reading device 1A, a plurality of cameras 5, 5A can be provided. One camera 5 can be a camera equipped with an optical system for photographing at a short distance, and the other camera 5A can be a camera equipped with an optical system for photographing at a longer distance than the one camera 5. Since the cameras 5, 5A have an autofocus function, a focused reading image can be generated from a short distance to a long distance.

[0054] When providing a plurality of cameras 5, 5A, the short-distance camera 5 can be a monochrome camera, and the long-distance camera 5A can be a color camera.

[0055] (Configuration of the processor) The following description is common to both the stationary optical information reading device 1 and the handheld optical information reading device 1A, and can be applied to either device 1 or 1A without special notice. As shown in FIG. 3, the processor 20 is composed of a multi-core processor having a plurality of cores that are central processing units. Specifically, the processor 20 includes a first general-purpose core 21, a second general-purpose core 22, a third general-purpose core 23, a fourth general-purpose core 24, and a dedicated core 25 dedicated to inference processing by a neural network. The first to fourth general-purpose cores 21 to 24 and the dedicated core 25 are a so-called System-on-a-chip (SoC, SOC) and are mounted on the same substrate. Note that the first to fourth general-purpose cores 21 to 24 and the dedicated core 25 do not have to be an SoC, and in that case, they do not have to be mounted on the same substrate, but this case is also included in the scope of the present invention. In this embodiment, the case where the number of general-purpose cores is four will be described, but the present invention is not limited to this, and the number of general-purpose cores may be one, or any number of two or more (for example, 6 cores, 8 cores, etc.).

[0056] A RAM 41 as the same memory is connected to the first to fourth general-purpose cores 21 to 24 and the dedicated core 25, and any of the first to fourth general-purpose cores 21 to 24 and the dedicated core 25 can access the same RAM 41. Further, a ROM 40 as the same memory is connected to the processor 20, and any of the first to fourth general-purpose cores 21 to 24 and the dedicated core 25 can also access the same ROM 40.

[0057] The first to fourth general-purpose cores 21 to 24 are so-called general-purpose processors, and are parts that execute, for example, AF control, illumination control, camera control, extraction processing for extracting a code candidate region, decoding processing for a read image, various filter processes for a read image, and the like. Specific processing examples of the first to fourth general-purpose cores 21 to 24 will be described later.

[0058] On the one hand, the dedicated core 25 is a core for performing super-resolution processing on the read image or performing inference processing for generating an ideal image corresponding to the read image using a neural network, and is specialized for executing the multiplication and accumulation operations required for the processing by the neural network at ultra-high speed. The dedicated core 25 includes, for example, an IC, an FPGA, or the like. Note that by applying the learning result by the neural network to perform inference processing, the read image can be restored to an image suitable for decoding, and thus generating an ideal image by the inference processing is also called restoring the read image. In this case, the dedicated core 25 is a part that attempts to restore the read image using a neural network.

[0059] As shown in FIG. 7, the processor 20 constitutes an AF control unit 20a, an imaging control unit 20b, a filter processing unit 20c, a tuning execution unit 20d, a decoding processing unit 20f, an extraction unit 20g, a reduction unit 20h, an enlargement unit 20i, an inference processing unit 20j, and a super-resolution processing unit 20k. The AF control unit 20a, the imaging control unit 20b, the filter processing unit 20c, the tuning execution unit 20d, the decoding processing unit 20f, the extraction unit 20g, the reduction unit 20h, and the enlargement unit 20i are parts constituted by the arithmetic processing of the first to fourth general-purpose cores 21 to 24. On the other hand, the inference processing unit 20j and the super-resolution processing unit 20k are parts constituted by the dedicated core 25.

[0060] (Configuration of AF control unit) The AF control unit 20a is a unit that controls the AF module 5c shown in FIG. 3, and is configured to be able to perform focusing of the optical system 5b by conventional contrast AF or phase difference AF. The AF control unit 20a may be constituted by cores that become the decoding processing unit 20f and the extraction unit 20g among the first to fourth general-purpose cores 21 to 24, or may be constituted by cores other than the cores that become the decoding processing unit 20f and the extraction unit 20g.

[0061] (Configuration of imaging control unit) The imaging control unit 20b is a unit that adjusts the gain of the camera 5, controls the light amount of the illumination unit 4, and controls the exposure time (shutter speed) of the imaging device 5a. Here, the gain of the camera 5 refers to the amplification factor (also called magnification) when amplifying the brightness of the image output from the imaging device 5a by digital image processing. Regarding the light amount of the illumination unit 4, the first illumination unit 4a and the second illumination unit 4b can be controlled and changed separately. The gain, the light amount of the illumination unit 4, and the exposure time are imaging conditions of the camera 5. The imaging control unit 20b may be composed of cores that become the decode processing unit 20f and the extraction unit 20g among the first to fourth general-purpose cores 21 to 24, or may be composed of cores other than the cores that become the decode processing unit 20f and the extraction unit 20g. Note that the AF control unit 20a and the imaging control unit 20b may be composed of the same core or different cores.

[0062] (Configuration of the filter processing unit) The filter processing unit 20c is a part that executes an image processing filter on the read image, and may be composed of cores that become the decode processing unit 20f and the extraction unit 20g, or may be composed of cores other than the cores that become the decode processing unit 20f and the extraction unit 20g. The core constituting the filter processing unit 20c may be a DSP core.

[0063] The filter processing unit 20c executes a noise removal filter that removes noise included in the image generated by the camera 5, a contrast correction filter that corrects contrast, an averaging filter, and the like. The image processing filter executed by the filter processing unit 20c is not limited to the noise removal filter, the contrast correction filter, and the averaging filter, and may include other image processing filters.

[0064] The filter processing unit 20c is configured to execute an image processing filter on the read image before the super-resolution processing and the inference processing, which will be described later. Further, the filter processing unit 20c is configured to execute an image processing filter on the read image before enlargement and reduction, which will be described later. Note that the filter processing unit 20c may be configured to execute an image processing filter on the read image after the super-resolution processing and the inference processing, or may be configured to execute an image processing filter on the read image after enlargement and reduction.

[0065] (Configuration of Tuning Execution Unit) The tuning execution unit 20d shown in FIG. 5 is a part that sets various conditions (tuning parameters) so that imaging conditions such as the gain of the camera 5, the light amount of the illumination unit 4, and the exposure time, and the image processing conditions in the filter processing unit 20c are changed to conditions suitable for decoding when the optical information reading devices 1 and 1A are set. The image processing conditions in the filter processing unit 20c include the coefficients of the image processing filter (the strength of the filter), the switching of the image processing filter when there are a plurality of image processing filters, the combination of different types of image processing filters, and the like. Appropriate imaging conditions and image processing conditions differ depending on the influence of external light on the workpiece W during conveyance, the color and material of the surface to which the code is attached, and the like. Therefore, the tuning execution unit 20d searches for more appropriate imaging conditions and image processing conditions, and sets the processing by the AF control unit 20a, the imaging control unit 20b, and the filter processing unit 20c.

[0066] Further, the tuning execution unit 20d includes a PPC value acquisition unit 20n. The PPC value acquisition unit 20n is a part that acquires the PPC value of the code included in the read image generated by the camera 5 at the time of setting the optical information reading device 1, and is configured to be able to acquire the size of the code included in the read image generated by the camera 5. When acquiring the size of the code, first, the PPC value acquisition unit 20n searches for the code based on the feature amount indicating the appearance of the code. Next, the tuning execution unit 20d acquires the PPC, code type, code size, etc. of the searched code. PPC, code type, code size, etc. are included in the code parameters or code conditions. Therefore, the PPC value acquisition unit 20n is configured to be able to acquire the code parameters or code conditions of the searched code.

[0067] The two-dimensional code is composed of a plurality of white modules and black modules randomly arranged. For example, in the case of QR code model 2, the number of modules ranges from 21×21 to 177×177. In the case of the stationary optical information reading device 1, by reading the code to be read in advance, the number of modules and PPC are limited, and the size of the code (pixel size) is limited to the number of modules × PPC for operation.

[0068] Among the modules constituting the code, when focusing on one module, the code parameter indicating how many pixels the module is composed of is the PPC. The PPC value acquisition unit 20n can acquire the PPC by counting the number of pixels constituting one module after specifying one module.

[0069] The code type is the type of code such as QR code, Data Matrix code, and Vericode. Since each code has characteristics, the tuning execution unit 20d can discriminate the code type by, for example, the presence or absence of a finder pattern.

[0070] Also, the code size can be calculated based on the number of modules arranged in the vertical or horizontal direction of the code and the PPC. The PPC value acquisition unit 20n can calculate the number of modules arranged in the vertical or horizontal direction of the code, calculate the PPC, and obtain the code size by multiplying the number of modules arranged in the vertical or horizontal direction by the PPC.

[0071] (Configuration of the decoding processing unit) The decoding processing unit 20f is a part that decodes black-and-white binarized data. A table showing the correspondence relationship of the encoded data can be used for decoding. Further, the decoding processing unit 20f checks whether the decoded result is correct according to a predetermined checking method. When an error is found in the data, the error correction function is used to calculate the correct data. The error correction function varies depending on the type of code.

[0072] In this embodiment, the decoding processing unit 20f decodes the code included in the enlarged image after the super-resolution processing described later. Also, the decoding processing unit 20f decodes the code included in the ideal image after the inference processing. Note that the decoding processing unit 20f can also decode the code included in the read image before the super-resolution processing and the read image before the inference processing. The decoding processing unit 20f is configured to write the decoding result obtained by decoding the code into the decoding result storage unit 40b of the ROM 40 shown in FIG. 3.

[0073] (Configuration of the extraction unit) The extraction unit 20g is a part that extracts a code candidate region where a code is likely to exist from the captured images generated by the camera 5. The code candidate region can be extracted based on a feature amount indicating code-likeness, and in this case, the feature amount indicating code-likeness becomes information for specifying the code. For example, the extraction unit 20g can acquire a captured image and search for a code in the acquired captured image based on a feature amount indicating code-likeness. Specifically, it searches whether there is a portion in the acquired captured image that has a feature amount indicating code-likeness equal to or more than a predetermined value. As a result, if a portion having a feature amount indicating code-likeness can be searched, the region including that portion is extracted as the code candidate region. The code candidate region may include regions other than the code, but at least includes a portion where the possibility of being a code is equal to or more than a predetermined value. Note that since the code candidate region is merely a region where a code is likely to exist, there may be a case where the region does not actually contain a code as a result.

[0074] The extraction unit 20g can also extract a code candidate area by using the position information of the Eimer light irradiated from the Eimer light irradiation unit 10 as information for specifying a code. In this case, for example, as shown in FIG. 8, the extraction unit 20g specifies, by image processing, a portion corresponding to a mark formed by the Eimer light 10d irradiated from the Eimer light irradiation unit 10 from among the read images within the visual field 5d of the camera 5, and extracts the specified portion as the code candidate area. That is, as described above, since the Eimer light 10d is for indicating the shooting range of the camera 5, the center of the visual field 5d, and the vicinity thereof, the mark formed by the Eimer light 10d can be positioned in advance within the visual field 5d of the camera 5. In particular, in the case of the handheld optical information reading device 1A, since the user aligns the Eimer light 10d with the code to be read, the mark formed by the Eimer light 10d has a high probability of overlapping with the code. That is, by positioning the portion corresponding to the mark formed by the Eimer light irradiation unit 10 within the visual field 5d of the camera 5, when the extraction unit 20g extracts the area corresponding to the Eimer light 10d within the visual field 5d of the camera 5, that area is a region where there is a high possibility that a code exists. In this case, it is not necessary to completely match the portion corresponding to the mark formed by the Eimer light irradiation unit 10 with the center of the visual field of the camera 5, and it may be slightly shifted in the vertical or horizontal direction of the visual field 5d, for example, as shown in FIG. 6. Also, since the code has a predetermined size, the area extracted by the extraction unit 20g is not only the center of the visual field 5d of the camera 5 but also an area having a predetermined size including the center of the visual field 5d.

[0075] As shown in FIG. 6, the position within the field of view 5d of the aiming light 10d changes depending on the distance between the handheld optical information reading device 1A and the code. Specifically, it changes depending on the distance between the light-receiving surface of the imaging element 5a of the camera 5 and the code. Therefore, the relationship between the distance between the optical information reading device 1A and the code and the position within the field of view 5d of the aiming light 10d is calculated in advance and stored in the storage device 30 or the like. During operation, the distance between the optical information reading device 1A and the code is obtained using a well-known distance sensor, the focus information of the camera 5, or the like, and the position within the field of view 5d of the aiming light 10d can be calculated based on the obtained distance.

[0076] The extraction unit 20g can also search for the finder pattern of the code included in the read image and extract the code candidate region based on the search result. For example, the finder pattern is searched for by image processing from the read image including the code, and a region within a predetermined range including the finder pattern is determined as the code candidate region, and the region is used as a partial image corresponding to the code candidate region.

[0077] The extraction unit 20g can also identify the central portion of the read image and extract the central portion as the code candidate region. For example, by previously obtaining the center of the field of view of the camera 5 as information for identifying the code, the portion corresponding to the center of the field of view of this camera 5 can be identified on the read image. In particular, in the case of the handheld optical information reading device 1A, since the central portion identified on the read image corresponds to the mark formed by the aiming light irradiation unit 10, the central portion is a region where there is a high possibility that the code exists.

[0078] The extraction unit 20g can be configured to receive a designation of a predetermined portion by the user from the captured image, and can extract the designated portion as a code candidate region. That is, when the user designates a region of an arbitrary size at an arbitrary position (coordinate) on the captured image, the extraction unit 20g receives the designation of a predetermined portion from the captured image based on the coordinate and size information of that position. The extraction unit 20g extracts the received portion as a code candidate region. For example, in the case of the stationary optical information reading device 1, as shown in FIG. 1, the work W being conveyed by the conveyor belt B is photographed and the code decoding process is executed. However, the work W does not necessarily exist at the center in the width direction of the conveyor belt B, and the work W may exist at the edge. In this case, by designating the portion corresponding to the edge of the conveyor belt B from the captured image, accurate extraction can be performed as a region where there is a high possibility that a code exists. In addition, there may be a case where a large work W has a code displayed near the edge away from the center. In this case, by designating the portion corresponding to the vicinity of the edge of the work W from the captured image, accurate extraction can be performed as a region where there is a high possibility that a code exists.

[0079] (Configuration of storage device, ROM) The storage device 30 shown in FIG. 3 can be configured by a readable and writable storage device such as an SSD (Solid State Drive). The storage device 30 can store various programs, setting information, image data, etc.

[0080] The ROM 40 includes an image data storage unit 40a, a decoding result storage unit 40b, a parameter set storage unit 40c, and a neural network storage unit 40d. The image data storage unit 40a is a part that stores the captured image generated by the camera 5, the enlarged image generated by super-resolution processing, the ideal image generated by inference processing, and the like. The decoding result storage unit 40b is a part that stores the decoding result of the code executed by the decoding processing unit 20f. The parameter set storage unit 40c is a part that stores the various conditions set as a result of the tuning executed by the tuning execution unit 20d, the code information including the PPC value, and the various conditions set by the user. The neural network storage unit 40d is a part that stores the structure and parameters of the learned neural network described later.

[0081] At least a part of the storage units 40a, 40b, 40c, and 40d may be provided in a part other than the ROM 40. For example, it may be provided in the storage device 30.

[0082] (Processing Using Neural Network) The optical information reading devices 1 and 1A have an enlarged image generation function that generates an enlarged image by performing super-resolution processing on the captured image generated by the camera using a neural network, and an ideal image generation function that generates an ideal image by performing inference processing on the captured image acquired by the camera 5 using a neural network. In the present embodiment, the ideal image generation function is not essential.

[0083] In the present embodiment, instead of performing machine learning with the optical information reading devices 1 and 1A, the optical information reading devices 1 and 1A previously hold the structure and parameters of the neural network that has already completed learning, and perform inference processing using the neural network configured with the held structure and parameters. An example of this configuration will be described, but the present invention is not limited to this, and machine learning may be performed with the optical information reading devices 1 and 1A to change the structure and parameters of the neural network.

[0084] As shown in FIG. 9, a neural network has an input layer into which input data (image data in this example) is input, an output layer that outputs output data, and an intermediate layer provided between the input layer and the output layer. A plurality of intermediate layers can be provided, for example, whereby a neural network having a multi-layer structure can be formed.

[0085] (Learning of Neural Network) First, the basic procedure during the learning of a neural network will be described based on the flowchart shown in FIG. 10. The learning of the neural network can be performed using a computer prepared for learning other than the optical information reading devices 1 and 1A, but it may also be performed using a general-purpose computer other than for learning. Further, learning may be performed by the optical information reading devices 1 and 1A. A conventionally well-known method may be used as the learning method of the neural network.

[0086] In step SA1 after starting, data of defective images prepared in advance is read. A defective image is an image having a portion inappropriate for code reading, and an image as shown on the left side of FIG. 11 can be exemplified. The inappropriate portion is, for example, a dirty portion or a colored portion, etc. Thereafter, the process proceeds to step SA2, and data of ideal images prepared in advance is read. An ideal image is an image appropriate for code reading, and an image as shown on the right side of FIG. 11 can be exemplified. The defective image and the ideal image are set as a pair, and a plurality of such pairs are prepared in advance. When learning is performed by the optical information reading devices 1 and 1A, the defective image and the ideal image are input to the optical information reading devices 1 and 1A.

[0087] In step SA2, a loss function is calculated to obtain the difference between the defective image and the ideal image. In step SA3, the parameters of the neural network are updated by reflecting the difference obtained in step SA2.

[0088] In step SA4, it is determined whether the completion condition of machine learning is satisfied. The completion condition of machine learning can be set based on the difference obtained in step SA2. For example, if the difference is equal to or less than a predetermined value, it can be determined that the completion condition of machine learning is satisfied. If it is determined as YES in step SA4 and the completion condition of machine learning is satisfied, the machine learning is terminated. On the other hand, if it is determined as NO in step SA4 and the completion condition of machine learning is not satisfied, the process returns to step SA1. After reading another defective image, the process proceeds to step SA2 and reads another ideal image paired with the other defective image.

[0089] In this way, by performing machine learning on a plurality of defective images and a plurality of ideal images respectively corresponding to the plurality of defective images, a pre-trained neural network can be generated. By having the optical information reading devices 1 and 1A hold the structure and parameters of the pre-trained neural network, a pre-trained neural network can be constructed within the optical information reading devices 1 and 1A. As in this example, if machine learning is performed outside the optical information reading devices 1 and 1A and the optical information reading devices 1 and 1A hold the structure and parameters of the neural network obtained as a result, inference processing by the neural network becomes possible while realizing downsizing and weight reduction of the optical information reading devices 1 and 1A. Incidentally, if a processor 20 with a sufficiently high arithmetic processing ability is mounted, there is no problem even if learning is performed by the optical information reading devices 1 and 1A.

[0090] Also, when training a neural network capable of performing super-resolution processing, for example, a low-resolution image is prepared as an input image (defective image), and machine learning is performed such that a high-resolution image is output by inputting this input image into the neural network. In this case, as the loss function of the neural network, the mean squared error between the image obtained by super-resolving the low-resolution image and the original high-resolution image (teacher image) can be used, and the parameters of the neural network are changed so that this error becomes smaller. The neural network trained in this way becomes capable of super-resolution processing, and thus can output an enlarged image obtained by enlarging the PPC indicating the number of pixels of each module constituting the code included in the captured image generated by the camera 5. The neural network for super-resolution processing can be specified by the structure and parameters of the neural network. As the neural network for super-resolution processing, for example, FSRCNN can be used.

[0091] As the defective image used when training the neural network for super-resolution processing, for example, an image including a code with a PPC within a certain range can be used. The certain range is, for example, PPC being 1.2 to 2.0. Also, the tilt angle at the time of capturing the defective image can be limited to substantially horizontal and substantially vertical, but a tilt angle of about plus or minus 10° may be attached.

[0092] FIG. 12 is a conceptual diagram showing an example of a convolutional neural network (CNN). In "Convolution", features of the input image are extracted. This layer is composed of a convolution operation like an image processing filter. The weights of the filter are called kernels, and feature amounts are extracted according to the kernels. The convolution layer generally has a plurality of different kernels, and the number of maps (D) increases according to the number of kernels. The kernel size can be, for example, 3×3, 5×5, etc.

[0093] In "pooling", reduction processing is performed to summarize the responses of each kernel. In "Deconvolution", an image is reconstructed from feature values using a transposed convolution filter. In "Unpooling", expansion processing is performed to sharpen the response values.

[0094] The structure and parameters of the learned neural network are stored in the neural network storage unit 40d of the ROM 40 shown in FIG. 3. The structure of the neural network refers to the number of intermediate layers (D) provided between the input layer and the output layer, the number of image processing filters, etc. The parameters of the neural network are the parameters set in step SA3 of the flowchart shown in FIG. 7.

[0095] In addition, the neural network storage unit 40d can store the structures and parameters of a plurality of neural networks with different numbers of layers or image processing filters. In this case, the structures and parameters of the plurality of neural networks and the code conditions can be stored in the neural network storage unit 40d in an associated state. For example, the structure and parameters of the neural network for constructing the first neural network are associated with the first code condition corresponding to the first neural network. When performing the association, since the number of layers or filters of the first neural network can be specified in advance as an optimal number by the first code condition, particularly the first PPC, the association is made so that this relationship is maintained. Similarly, the structure and parameters of a second neural network different from the first neural network are associated with a second code condition different from the first code condition.

[0096] When the structures and parameters of a plurality of neural networks are stored in the neural network storage unit 40d, the tuning execution unit 20d operates as follows when setting the optical information reading devices 1 and 1A. That is, when setting the code conditions included in the read image generated by the camera 5, the tuning execution unit 20d reads out the structures and parameters of the neural network associated with the set code conditions from the neural network storage unit 40d and specifies them as the neural network to be used during operation. For example, when the tuning execution unit 20d sets the first code condition by setting the code conditions included in the read image, it reads out the structure and parameters of the first neural network associated with the first code condition from the neural network storage unit 40d. Then, it specifies the structure and parameters of the first neural network as the neural network to be used during the operation of the optical information reading devices 1 and 1A. Thereby, since the inference process can be executed on the read image with the structure and parameters of the neural network optimal for the code conditions, the reading accuracy can be improved.

[0097] In the learning of the neural network, it is also possible to perform machine learning on defective images and ideal images in which the pixel resolution of the modules constituting the code is within a specific range. The pixel resolution of the module can be represented, for example, by the number of pixels (PPC) of one module constituting the code, and the neural network can be machine-learned only with defective images and ideal images in which the PPC is within the specific range.

[0098] The above specific range includes the pixel resolution with a high inference processing effect on the read image and excludes the pixel resolution with almost no improvement in the inference processing effect on the read image, and can be set on the premise of the inference processing on an image including a code composed of a plurality of modules. Thereby, the processing speed can be improved as a neural network structure specialized for the inference processing on an image including a code.

[0099] As the specific range of the number of pixels, for example, it can be 4 PPC or more and 6 PPC or less, but it may also be less than 4 PPC or more than 6 PPC. Further, for example, by limiting the specific range to 4 PPC or more and 6 PPC or less and performing machine learning, it is possible to eliminate unnecessary scale variations from the images used for learning, and it is also possible to optimize the learned neural network structure.

[0100] (Inference processing) As shown in FIG. 7, the processors 20 of the optical information reading devices 1 and 1A are provided with an inference processing unit 20j constituted by a dedicated core 25. The inference processing unit 20j reads out the structure and parameters of the neural network stored in the neural network storage unit 40d of the ROM 40, and configures a neural network for inference processing in the optical information reading devices 1 and 1A with the read structure and parameters of the neural network. The neural network configured in the optical information reading devices 1 and 1A is the same as the learned neural network learned according to the procedure shown in the flowchart of FIG. 10.

[0101] The inference processing unit 20j executes an inference process of generating an ideal image corresponding to the read image according to the structure and parameters of the neural network stored in the neural network storage unit 40d by inputting the read image generated by the camera 5 into the neural network. Since the ideal image generated by the inference processing unit 20j is merely an image generated by the inference process, it is not necessarily the same as the ideal image used during the above-described learning. However, in the present embodiment, the image generated by the inference processing unit 20j is also referred to as an ideal image.

[0102] The filter processing unit 20c is configured to execute an image processing filter on the read image before the inference processing unit 20j inputs the read image generated by the camera 5 into the neural network. Thereby, appropriate image processing can be executed before the inference processing by the neural network, and as a result, the inference processing becomes more accurate.

[0103] The inference processing unit 20j may perform inference processing on all the read images decoded by the decoding processing unit 20f, or may perform inference processing only on some of the read images decoded by the decoding processing unit 20f. For example, the decoding processing unit 20f performs decoding processing on the read image before performing inference processing, and determines whether the decoding is successful. If the decoding is successful, it means that the read image does not require inference processing, and the result is output as it is. On the other hand, if the decoding fails, the inference processing unit 20j performs inference processing, and the decoding processing unit 20f performs decoding processing on the generated ideal image.

[0104] (Super-resolution processing) As shown in FIG. 7, the processors 20 of the optical information reading devices 1 and 1A are provided with a super-resolution processing unit 20k constituted by a dedicated core 25. The super-resolution processing unit 20k reads out the structure and parameters of the neural network stored in the neural network storage unit 40d of the ROM 40, and configures a neural network for super-resolution processing in the optical information reading devices 1 and 1A with the read structure and parameters of the neural network. The neural network for super-resolution processing configured in the optical information reading devices 1 and 1A is the same as the learned neural network learned with the above-described low-resolution image and high-resolution image.

[0105] The super-resolution processing unit 20k inputs the read image generated by the camera 5 into the neural network for super-resolution processing, and generates an enlarged image in which the PPC value of the read image is enlarged according to the structure and parameters of the neural network stored in the neural network storage unit 40d. That is, the enlarged image generated by the super-resolution processing unit 20k is an image in which the neural network estimates high-frequency components not included in the original read image instead of interpolation between pixels, and thereby the PPC value of the read image can be enlarged.

[0106] For example, if the PPC value of the code included in the captured image generated by the camera 5 is less than 2 (for example, about 1.2 to 1.5), the decoding process may not be possible at all. However, by increasing the PPC value to 2 or more through the super-resolution processing by the super-resolution processing unit 20k, the decoding process becomes possible. In the present embodiment, even if the PPC value is 0.1, the decoding process becomes possible by applying the super-resolution processing.

[0107] Also, if the PPC value of the code included in the captured image generated by the camera 5 is less than 1.5 to 2.0, the decoding process is barely possible, but since the decoding process is not stable, by increasing the PPC value in this range to 2.0 or more, a stable decoding process becomes possible.

[0108] As the PPC value after enlargement, for example, it can be 2.2 or more, or 2.4 or more. The PPC value after enlargement can be set according to the enlargement ratio by the neural network. The enlargement ratio by the neural network can be any integer multiple, for example, 2.0 times, 3.0 times, etc. If the enlargement ratio by the neural network is 2.0 times, assuming that the size of the captured image input to the neural network is 128×128 (pixels), the size of the output enlarged image will be 256×256 (pixels).

[0109] The super-resolution processing unit 20k is configured to generate an enlarged image when the PPC value acquired by the PPC value acquisition unit 20n is less than or equal to a predetermined value, and not to generate an enlarged image when the PPC value acquired by the PPC value acquisition unit 20n exceeds the predetermined value. The predetermined value can be, for example, any value between 2.0 and 2.3. As described above, if the PPC value exceeds 2, the decoding process becomes stably possible, so the need for an enlarged image is low. In this case, by not executing the super-resolution processing, the processing speed can be improved.

[0110] Also, in the handheld optical information reading device 1A shown in FIG. 4, it has a camera 5 for short distances and a camera 5A for long distances, and each camera 5, 5A can generate a focused reading image from short distances to long distances. In this case, super-resolution processing may be executed on both the reading image generated by the short-distance camera 5 and the reading image generated by the long-distance camera 5A. Thereby, an enlarged image can be generated from a reading image including a code at a long distance or a small code that is difficult to read even when photographed at a short distance.

[0111] Also, when the short-distance camera 5 is a monochrome camera and the long-distance camera 5A is a color camera, after converting the color reading image generated by the long-distance camera 5A into monochrome, super-resolution processing may be executed. In this case, the processor 20 has a configuration including a monochrome conversion unit.

[0112] The filter processing unit 20c is configured to execute an image processing filter on the reading image before inputting the reading image generated by the camera 5 to the neural network for super-resolution processing. Thereby, appropriate image processing can be executed before executing the super-resolution processing by the neural network for super-resolution processing, and as a result, the enlargement processing becomes clearer.

[0113] The super-resolution processing unit 20k may execute super-resolution processing on all the reading images to be decoded by the decode processing unit 20f, or the super-resolution processing unit 20k may execute super-resolution processing only on some of the reading images to be decoded by the decode processing unit 20f. For example, the decode processing unit 20f executes decode processing on the reading image before executing super-resolution processing, and determines whether the decode is successful. If the decode is successful, it means that the reading image does not require super-resolution processing, and the result is output as it is. On the other hand, if the decode fails, the super-resolution processing unit 20k executes super-resolution processing, and the decode processing unit 20f executes decode processing on the generated enlarged image.

[0114] (Input of partial image) The images input by the inference processing unit 20j and the super-resolution processing unit 20k to the neural network may be all of the captured images generated by the camera 5. However, the larger the size of the input image, the greater the amount of computation by the neural network, the heavier the computational load on the processor 20, and ultimately, there is a risk of a decrease in the processing speed. Specifically, in the case of a FCN (Fully Convolutional Network) type neural network as shown in FIG. 12, assuming the structures are equal, the amount of computation is proportional to the input image size. Since the input image size is proportional to the square of the code size, the amount of computation is also proportional to the square of the code size.

[0115] On the other hand, it can be said that there is no case where codes exist in the entire captured image generated by the camera 5. During normal operation, codes exist only in a partial region of the captured image. If inference processing by the neural network can be executed only on that region, the reading accuracy can be improved.

[0116] Therefore, the images input by the inference processing unit 20j and the super-resolution processing unit 20k to the neural network can be a part of the captured image generated by the camera 5, that is, a partial image including the code. Specifically, the partial image corresponding to the code candidate region extracted by the extraction unit 20g is stored in the image data storage unit 40a and made accessible to the inference processing unit 20j and the super-resolution processing unit 20k. The inference processing unit 20j and the super-resolution processing unit 20k input the partial image to the neural network and execute inference processing and super-resolution processing on the partial image according to the structure and parameters stored in the neural network storage unit 40d. Also, the inference processing by the inference processing unit 20j may be executed after the super-resolution processing by the super-resolution processing unit 20k, and then the decoding processing unit 20f may execute decoding processing.

[0117] Then, the decoding processing unit 20f performs decoding processing on the generated ideal image or enlarged image. Also, as will be described later, when performing decoding processing without performing inference processing and super-resolution processing on the partial image, the decoding processing unit 20f is made accessible to the image data storage unit 40a, and the decoding processing unit 20f reads the partial image stored in the image data storage unit 40a and performs decoding processing.

[0118] As described above, there are almost no cases where codes exist in the entire read image. Therefore, the size of the partial image corresponding to the code candidate region is smaller than the size of the read image. As a result, since the size of the image input to the neural network becomes smaller, the computational load on the processor 20 is reduced, and thus the processing speed can be increased. Also, since the partial image corresponds to a region where there is a high possibility of the existence of a code, it is possible to obtain an image including all the information necessary for reading, that is, the entire code. Since the image including the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

[0119] The inference processing unit 20j and the super-resolution processing unit 20k can determine the input size of the partial image to the neural network based on the code size acquired by the tuning execution unit 20d. As described above, the tuning execution unit 20d can acquire the code size. The inference processing unit 20j and the super-resolution processing unit 20k make the input size of the partial image input to the neural network larger by a predetermined amount than the code size acquired by the tuning execution unit 20d.

[0120] That is, by inputting a partial image into a neural network, the computational load can be reduced. However, for example, if the code is rotated or the position of the code cannot be precisely detected, a part of the code in the partial image may be missing, and the decoding process may fail. In the present embodiment, the input size to the neural network is not the same as the code size acquired by the tuning execution unit 20d, but is larger than the code size by a predetermined amount. Therefore, it is possible to suppress a part of the code from being missing in the partial image. By making the size larger than the code size by a predetermined amount, an upper limit can be set for the input size to the neural network, and thus the computational load can be prevented from becoming heavy. The "predetermined amount" may be, for example, that the horizontal (or vertical) length of the partial image input to the neural network is 1.5 times or more, or 2.0 times or more, or 3.0 times or more the horizontal (or vertical) length of the code acquired by the tuning execution unit 20d.

[0121] When determining the input size of the partial image input to the neural network by the inference processing unit 20j and the super-resolution processing unit 20k, it can be determined based on the number of pixels of one module constituting the code acquired by the tuning execution unit 20d and the number of modules arranged in the vertical or horizontal direction of the code.

[0122] In addition, as described above, the tuning execution unit 20d can set the code conditions. The inference processing unit 20j and the super-resolution processing unit 20k can also set the size of the read image input to the neural network based on the code conditions set by the tuning execution unit 20d. For example, when the inference processing unit 20j and the super-resolution processing unit 20k acquire the code size among the code conditions set by the tuning execution unit 20d, a partial image having a size larger than the acquired code size by a predetermined amount is input to the neural network. If the code size is large, the size of the partial image input to the neural network becomes large, while if the code size is small, the size of the partial image input to the neural network becomes small. That is, the size of the partial image input to the neural network can be changed according to the code conditions.

[0123] (Shrinking / Enlarging Function) Here, in order to sufficiently capture the features of the code in the convolutional neural network as shown in FIG. 12 and obtain an ideal image suitable for decoding, it is necessary to extract feature amounts within a range covering a certain number of modules. For example, assuming that one feature value obtained from the neural network is a value extracted from a range of 6×6 modules, and one module is an image photographed at 20 pixels, the neural network must be designed to calculate a feature value aggregated from a range of 120×120 pixels. That is, the wider the pixel range to be covered, the deeper the neural network hierarchy must be. For example, if we want to aggregate feature values from the above-mentioned range of 120×120 pixels, six convolutional layers are required.

[0124] However, since the code is composed of randomly arranged white and black modules, the relationship of pixel values in a wide range is sparse, and even if the pixel range to be covered is expanded beyond a certain level, the inference processing effect hardly improves. Such a feature is referred to as a narrow-range feature in this specification.

[0125] In addition, the arrangement of fixed module patterns such as finder patterns included in the code and the fact that the entire code is square-shaped can be broad-range features. However, if it is only necessary to roughly separate the module from the background, inference processing can be sufficiently performed using only narrow-range features without using broad-range features. Note that the broad-range features can be defined as features having a fixed shape over a wide range.

[0126] Therefore, when training the neural network shown in FIG. 10, images in which the PPC of the code is within a specific range can be used. As a result, the processing speed can be improved as a neural network structure specialized for inference processing of images including the code. On the other hand, there is a concern that the inference processing effect for read images in which the PPC is outside the specific range may decrease. Note that when training the neural network, images in which the PPC of the code is outside the specific range may also be used.

[0127] In the optical information reading devices 1 and 1A of the present embodiment, in order to make the PPC of the read image fall within the specific range, a reduction / enlargement function of the read image is provided. As shown in FIG. 7, the processor 20 includes a reduction unit 20h and an enlargement unit 20i. The reduction unit 20h is a part that generates a reduced read image such that the pixel resolution of the module constituting the code in the read image generated by the camera 5 is within the above-described specific range. The enlargement unit 20i is a part that generates an enlarged read image such that the pixel resolution of the module constituting the code in the read image generated by the camera 5 is within the above-described specific range.

[0128] The reduction unit 20h and the enlargement unit 20i determine, for example, whether the PPC of the code in the read image generated by the camera 5 is outside the specific range (4PPC or more and 6PPC or less). If it is within the specific range, reduction or enlargement is not executed. On the other hand, if it is outside the specific range, reduction or enlargement is executed. By executing reduction or enlargement, the PPC of the code in the partial image input to the inference processing unit 20j and the super-resolution processing unit 20k into the neural network falls within the specific range.

[0129] The inference processing unit 20j and the super-resolution processing unit 20k input the read image enlarged or reduced so as to be within a specific range into a neural network configured with the structure and parameters stored in the neural network storage unit 40d, and execute inference processing on the read image according to the structure and parameters. Then, the decoding processing unit 20f executes decoding processing on the generated ideal image.

[0130] The extraction unit 20g may be configured to search for a code in the read image enlarged by the enlargement unit 20i or the read image reduced by the reduction unit 20h, and extract the region including the searched code as a code candidate region where the code is likely to exist. In this case, the inference processing unit 20j and the super-resolution processing unit 20k input the partial image corresponding to the code candidate region extracted by the extraction unit 20g into a neural network configured with the structure and parameters stored in the neural network storage unit 40d, and execute inference processing on the read image according to the structure and parameters.

[0131] When setting the optical information reading apparatuses 1 and 1A, the tuning execution unit 20d can generate a plurality of read images (enlarged images) with different enlargement ratios and a plurality of read images (reduced images) with different reduction ratios. The tuning execution unit 20d inputs the generated plurality of read images (enlarged images or reduced images) into the neural network, executes inference processing on each read image, executes decoding processing on each generated ideal image, and obtains a reading margin indicating the ease of code reading. The tuning execution unit 20d specifies the enlargement ratio or reduction ratio of the read image with a reading margin higher than a predetermined value as the enlargement ratio or reduction ratio to be used during operation. When the tuning execution unit 20d specifies the enlargement ratio or reduction ratio, during the operation of the optical information reading apparatuses 1 and 1A, the reduction unit 20h and the enlargement unit 20i enlarge or reduce the read image at the enlargement ratio or reduction ratio specified by the tuning execution unit 20d.

[0132] That is, for example, when a specific range has a certain width, even within that specific range, if the magnification or reduction ratio is changed, it is conceivable that the reading margin will change. Since the magnification or reduction ratio of the read image during operation can be specified so that the reading margin obtained by the tuning execution unit 20d is higher than a predetermined value, the processing speed and the reading accuracy are improved.

[0133] After the tuning execution unit 20d obtains the reading margin of each read image, among the plurality of obtained reading margins, the magnification or reduction ratio of the read image with the highest reading margin can be specified as the magnification or reduction ratio to be used during operation. Thereby, the processing speed and the reading accuracy can be further improved.

[0134] Also, by utilizing the fact that the reduction unit 20h and the enlargement unit 20i can reduce and enlarge the read image, the types of parameter sets can be increased. A plurality of parameter sets with different reduction ratios by the reduction unit 20h and a plurality of parameter sets with different enlargement ratios by the enlargement unit 20i can be generated, and these parameter sets can be stored in the parameter set storage unit 40c of the ROM 40. For example, when a plurality of reduction ratios such as 1 / 2, 1 / 4, and 1 / 8 can be set by the reduction unit 20h, a parameter set with a reduction ratio of 1 / 2, a parameter set with a reduction ratio of 1 / 4, and a parameter set with a reduction ratio of 1 / 8 can be stored in the parameter set storage unit 40c. Also, when a plurality of enlargement ratios such as 2 times, 4 times, and 8 times can be set by the enlargement unit 20i, a parameter set with an enlargement ratio of 2 times, a parameter set with an enlargement ratio of 4 times, and a parameter set with an enlargement ratio of 8 times can be stored in the parameter set storage unit 40c. And any one of the plurality of parameter sets stored in the parameter set storage unit 40c can be applied.

[0135] (Inference Processing Filter and Super-Resolution Processing Filter) The inference processing unit 20j may be a part that executes an inference processing filter that performs inference processing using a neural network. Also, the super-resolution processing unit 20k may be a part that executes a super-resolution processing filter that performs super-resolution processing using a neural network.

[0136] The inference processing filter is a filter that inputs the read image into a neural network composed of the structure and parameters stored in the neural network storage unit 40d, and executes inference processing according to the structure and parameters stored in the neural network storage unit 40d, and exhibits the same function as the above-described inference processing function.

[0137] Also, the super-resolution processing filter is a filter that inputs the read image into a neural network for super-resolution processing composed of the structure and parameters stored in the neural network storage unit 40d, and executes super-resolution processing according to the structure and parameters stored in the neural network storage unit 40d, and exhibits the same function as the above-described super-resolution processing function.

[0138] When it is possible to execute the inference processing filter or the super-resolution processing filter, a setting unit 20e (shown in FIG. 7) for setting the inference processing filter or the super-resolution processing filter can be provided. The setting unit 20e is configured to be able to receive the setting of the inference processing filter or the super-resolution processing filter by the user. The setting unit 20e can be configured to generate, for example, a user interface in which the user can select one of "application of the inference processing filter" and "non-application of the inference processing filter" and display it on the display unit 6 when setting the optical information reading devices 1, 1A, and receive the selection by the user. Also, the setting unit 20e can be configured to generate, for example, a user interface in which the user can select one of "application of the super-resolution processing filter" and "non-application of the super-resolution processing filter" and display it on the display unit 6 when setting the optical information reading devices 1, 1A, and receive the selection by the user.

[0139] The setting unit 20e may be included in the tuning execution unit 20d or may be configured separately from the tuning execution unit 20d. When the setting unit 20e is included in the tuning execution unit 20d, it becomes possible to set parameters related to the inference processing filter or the super-resolution processing filter during tuning. The parameters related to the inference processing filter can include "application of the inference processing filter" and "non-application of the inference processing filter". "Application of the inference processing filter" means setting the inference processing filter to be executable, and "non-application of the inference processing filter" means not setting the inference processing filter.

[0140] Also, the parameters related to the super-resolution processing filter can include "application of the super-resolution processing filter" and "non-application of the super-resolution processing filter". "Application of the super-resolution processing filter" means setting the super-resolution processing filter to be executable, and "non-application of the super-resolution processing filter" means not setting the super-resolution processing filter.

[0141] The parameters related to the inference processing filter or the super-resolution processing filter are also information related to the inference processing filter or the super-resolution processing filter. In this case, a parameter set including information related to the setting of the inference processing filter or the super-resolution processing filter can be stored in the parameter set storage unit 40c. The parameter set stored in the parameter set storage unit 40c can include a first parameter set for setting the inference processing filter to be executable, a second parameter set for not setting the inference processing filter, a third parameter set for setting the super-resolution processing filter to be executable, and a fourth parameter set for not setting the super-resolution processing filter.

[0142] During the operation of the optical information reading devices 1 and 1A, one parameter set selected from the first parameter set, the second parameter set, the third parameter set, and the fourth parameter set is applied. When the first parameter set is applied, decoding processing by the decoding processing unit 20f is executed on the ideal image on which inference processing has been executed by the inference processing filter. On the other hand, when the second parameter set is applied, decoding processing by the decoding processing unit 20f is executed on the read image on which inference processing is not executed. The same applies to the third parameter set and the fourth parameter set.

[0143] The parameter sets stored in the parameter set storage unit 40c also include items for setting the code conditions included in the read image generated by the camera 5. In the items for setting the code conditions, PPC, code type, code size, etc., acquired by the tuning execution unit 20d are set. For example, one parameter set can include PPC, code type, and code size as items for setting code conditions, the gain of the camera 5, the light quantity of the illumination unit 4, and the exposure time as imaging conditions, the type of image processing filter as an item of the image processing filter applied by the filter processing unit 20c, the parameters of the image processing filter, and the application items of the inference processing filter and the super-resolution processing filter. These respective items may use the values set by the tuning execution unit 20d as they are, or can be arbitrarily changed by the user.

[0144] (An example of the procedure of the tuning process) An example of the procedure of the tuning process performed by the tuning execution unit 20d when setting the optical information reading devices 1 and 1A will be specifically described based on the flowchart shown in FIG. 13. In step SB1 after the start of the flowchart shown in FIG. 13, the tuning execution unit 20d controls the illumination unit 4 and the camera 5 to generate a read image in the camera 5, and the tuning execution unit 20d acquires the read image. At this time, the presence or absence and type of the image processing filter executed before the decoding process, the inference process by the neural network, and the decoding process parameters for applying the super-resolution process are set to arbitrary parameters. Next, the process proceeds to step SB2, and the tuning execution unit 20d causes the decoding process unit 20f to execute the decoding process on the acquired read image.

[0145] After the decoding process, the process proceeds to step SB3, and the tuning execution unit 20d determines whether or not the decoding process in step SB2 was successful. If it is determined as NO in step SB3 and the decoding process in step SB2 fails, that is, if the code cannot be read, the process proceeds to step SB4. After changing the decoding process parameters to other parameters, the decoding process is executed again in step SB2. For example, if the parameters are such that the super-resolution process is not executed, the parameters are changed to those for which the super-resolution process is executed, and before step SB2, the super-resolution process is executed on the read image. If the decoding process fails with all the decoding process parameters, this flow is terminated and the user is notified.

[0146] On the other hand, if it is determined as YES in step SB3 and the decoding process in step SB2 is successful, the process proceeds to step SB5, and the tuning execution unit 20d determines the code parameters (PPC, code type, code size, etc.). When the tuning execution unit 20d determines the code parameters, the structure and parameters of the neural network stored in the neural network storage unit 40d associated with the code parameters are also determined (step SB6). In step SB6, the neural network is configured with the structure and parameters read from the neural network storage unit 40d.

[0147] In step SB7, the inference processing unit 20j executes inference processing by inputting the read image into the neural network configured in step SB6, and the decoding processing unit 20f executes decoding processing on the generated ideal image. Then, it proceeds to step SB8, and the tuning execution unit 20d evaluates the reading margin based on the decoding processing result of step SB7 and temporarily stores it.

[0148] In step SB9, it is determined whether the execution of the decoding process has been completed for all the decoding process parameters. If it is determined as NO in step SB9 and the execution of the decoding process has not been completed for all the decoding process parameters, it proceeds to step SB10. After changing the decoding process parameters to other parameters, it proceeds to step SB7.

[0149] On the other hand, if it is determined as YES in step SB9 and the execution of the decoding process has been completed for all the decoding process parameters, it proceeds to step SB11. In step SB11, the tuning execution unit 20d selects the decoding process parameter with the highest reading margin from among all the decoding process parameters and determines it as the parameter to be applied during operation. Incidentally, in the tuning process, imaging conditions and the like are also set to appropriate conditions.

[0150] (Decoding Process Procedure before Determining Reduction Ratio and Enlargement Ratio) Next, an example of the decoding process procedure before determining the reduction ratio and enlargement ratio will be specifically described based on the flowchart shown in FIG. 14. The process specified in the flowchart shown in FIG. 14 can be executed in step SB2 of the flowchart shown in FIG. 13.

[0151] In step SC1 after the start of the flowchart shown in FIG. 14, the tuning execution unit 20d selects an arbitrary image processing filter from among a plurality of image processing filters. This image processing filter is a filter executed by the filter processing unit 20c. Thereafter, in step SC2, the filter processing unit 20c executes the image processing filter selected in step SC1 on the read image.

[0152] Next, it proceeds to step SC3, and an image pyramid is created. The image pyramid is composed of the original read image, an image obtained by reducing the original read image by 1 / 2, an image obtained by reducing the original read image by 1 / 4, an image obtained by reducing the original read image by 1 / 8, and so on. The reduction of the read image is executed by the reduction unit 20h. Also, the image pyramid can be composed of the original read image, an image obtained by enlarging the original read image by 2 times, an image obtained by enlarging the original read image by 4 times, an image obtained by enlarging the original read image by 8 times, and so on. The enlargement of the read image is executed by the enlargement unit 20i. Note that either the reduced image or the enlarged image may be omitted.

[0153] After creating the image pyramid, proceed to step SC4 and select an arbitrary read image from among the plurality of read images that make up the image pyramid. This selected image may include the original read image that has not been reduced or enlarged. In step SC5, the extraction unit 20g extracts a code candidate region where a code is likely to exist from among the read images selected in step SC4. Thereafter, proceeding to step SC6, in the case of parameters for which super-resolution processing is to be executed, the super-resolution processing unit 20k inputs a partial image corresponding to the code candidate region extracted in step SC5 to a neural network and executes super-resolution processing. Also, in step SC6, in the case of parameters for which inference processing is to be executed, the inference processing unit 20j inputs a partial image corresponding to the code candidate region extracted in step SC5 to a neural network and executes inference processing. The size of the partial image input to the neural network is determined by the above-described code conditions. Note that in the case of parameters for which super-resolution processing is not executed, super-resolution processing is not executed in step SC6, and in the case of parameters for which inference processing is not executed, inference processing is not executed in step SC6.

[0154] After passing through step SC6, proceed to step SC7 and execute positioning processing for the contour of the code and the positions of the modules that make up the code. After the positioning processing, proceed to step SC8 and determine whether each module that makes up the code is white or black. After the black-and-white determination, proceed to step SC9, and the decoding processing unit 20f executes decoding processing on the generated ideal image. In the decoding processing, for example, a method of restoring a character string from the 0-1 matrix of the module can be adopted.

[0155] After that, the process proceeds to step SC10, where the tuning execution unit 20d determines whether the decoding process in step SC9 was successful. If it is determined as NO in step SC10 and the decoding process in step SC9 fails, the process proceeds to step SC11, where it is determined whether all the read images that make up the image pyramid have been selected. If it is determined as NO in step SC11 and not all the read images that make up the image pyramid have been selected, the process proceeds to step SC4, where another reduced or enlarged read image that makes up the image pyramid is selected, and the process proceeds to step SC5.

[0156] On the other hand, if it is determined as YES in step SC11 and all the read images that make up the image pyramid have been selected, this flow ends, and the conditions for successful decoding are stored.

[0157] (Decoding processing procedure after determining the reduction rate and the enlargement rate) Next, an example of the decoding processing procedure after determining the reduction rate and the enlargement rate will be specifically described based on the flowchart shown in FIG. 15. The processing specified in the flowchart shown in FIG. 15 can be executed during the operation of step SB7 of the flowchart shown in FIG. 13 and the optical information reading devices 1 and 1A.

[0158] In step SD1 after the start of the flowchart shown in FIG. 15, the filter processing unit 20c executes the image processing filter selected in step SC1 of the flowchart shown in FIG. 14 on the read image. At this time, if the filter processing unit 20c executes an averaging filter on the read image as shown in the upper part of FIG. 16, an image after executing the image processing filter as shown in the lower part of FIG. 16 is obtained.

[0159] After that, the process proceeds to step SD2, where reduction / enlargement processing of the read image is executed as necessary. Specifically, when the PPC of the code in the read image after the image processing filter is executed is within a specific range, reduction or enlargement is not executed. On the other hand, when it is outside the specific range, reduction or enlargement is executed so that the PPC falls within the specific range. An example of the read image after the reduction process is shown in the upper part of FIG. 17.

[0160] Next, in step SD3, the extraction unit 20g extracts a code candidate region where a code is likely to exist from the read image reduced or enlarged in step SD2. An example of the code candidate region extraction image is shown in the lower part of FIG. 17. Note that when reduction or enlargement has not been performed in step SD2, the extraction process is executed on the original read image in step SD3.

[0161] After that, the process proceeds to step SD4, where the inference processing unit 20j inputs the partial image corresponding to the code candidate region extracted in step SD3 into the neural network and executes inference processing. FIG. 18 shows examples of the read image before the inference processing and the ideal image after the inference processing. After the generation of the ideal image, this flow ends through steps SD5 to SD7. Steps SD5 to SD7 are the same as steps SC7 to SC9 in the flowchart shown in FIG. 14.

[0162] (First example during operation including super-resolution processing) A first example of executing super-resolution processing during the operation of the optical information reading devices 1 and 1A will be described based on the flowchart shown in FIG. 19. In step SE1 after starting, the extraction unit 20g extracts a code candidate region where a code is likely to exist from the read image generated by the camera 5. As shown in the image example in FIG. 20, the region surrounded by the rectangular frame in the read image is the code candidate region, and this code candidate region is extracted as a partial image.

[0163] Thereafter, the process proceeds to step SE2, where the filter processing unit 20c executes an image processing filter on the partial image. The image processing filter is a noise removal filter. Particularly during long-distance shooting where the camera 5 and the cord are far apart, the lighting may not reach the cord, and the camera 5 may increase the gain in automatic brightness control, resulting in an increase in random noise. If super-resolution processing is executed in a state with a lot of random noise, the noise may be emphasized and have an adverse effect on cell restoration. Therefore, by executing a noise removal filter before executing super-resolution processing to remove extra noise, the effect of super-resolution processing can be enhanced. Since a normal averaging filter may erase cell information as a noise removal filter, noise removal using, for example, L1 sparse modeling or the like is executed. Also, at this time, contrast adjustment is performed. The partial image after the image processing filter is shown on the lower side of FIG. 20. Steps SE1 and SE2 may be in the reverse order. That is, the code candidate region may be extracted after the execution of the image processing filter.

[0164] Next, the process proceeds to step SE3, where the super-resolution processing unit 20k executes super-resolution processing on the partial image after the image processing filter processing. As a result, as shown on the lower side of FIG. 21, an enlarged image is generated. In this example, a case where the magnification is 2 times is shown. The subsequent steps SE4 to SE6 are the same as steps SC7 to SC9 in the flowchart shown in FIG. 14.

[0165] (Second example during operation including super-resolution processing) A second example of performing super-resolution processing during the operation of the optical information reading devices 1 and 1A will be described based on the flowchart shown in FIG. 22. Before describing the second example, two types of decoding processes will be described. As described above, in the present embodiment, there are cases where the decoding processing unit 20f performs decoding processing on the read image without the super-resolution processing unit 20k performing super-resolution processing on the read image generated by the camera 5, and cases where the super-resolution processing unit 20k performs super-resolution processing on the read image generated by the camera 5 to generate an enlarged image, and the decoding processing unit 20f performs decoding processing on the generated enlarged image. In the former case, that is, the case of performing decoding processing on the read image is referred to as the first decoding process, and in the latter case, that is, the case of performing decoding processing on the enlarged image generated by the super-resolution processing is referred to as the second decoding process.

[0166] Both the first decoding process and the second decoding process are executed by the decoding processing unit 20f, but they may be executed by different decoding processing units. That is, since the processor 20 has the dedicated core 25 and the first to fourth general-purpose cores 21 to 24, the super-resolution processing by the super-resolution processing unit 20k composed of the dedicated core 25 and the first decoding process by the decoding processing unit 20f composed of at least one of the first to fourth general-purpose cores 21 to 24 can be executed in parallel and at high speed. Further, the processor 20 can execute the second decoding process by the decoding processing unit 20f after the completion of the super-resolution processing by the super-resolution processing unit 20k.

[0167] As an example, the first general-purpose core 21 can execute the first decoding process, and the second general-purpose core 22 can execute the second decoding process. Specifically, while the first general-purpose core 21 executes the first decoding process, the dedicated core 25 executes the super-resolution processing. When the super-resolution processing is completed, the second general-purpose core 22 executes the second decoding process without waiting for the completion of the first decoding process. Thereby, since the first decoding process and the second decoding process can be executed by different general-purpose cores, the processing becomes even faster.

[0168] Also, any one of the first to fourth general-purpose cores 21 to 24 of the processor 20 may be configured to execute the first decoding process and the second decoding process in parallel in separate threads. Specifically, for example, while the first general-purpose core 21 executes the first decoding process, the dedicated core 25 executes the super-resolution process. When the super-resolution process ends, the first general-purpose core 21 executes the second decoding process in a separate thread from the first decoding process without waiting for the end of the first decoding process.

[0169] In step SF1 after the start of the flowchart shown in FIG. 22, the processor 20 acquires a read image. Thereafter, in step SF2, the first decoding process is performed, that is, the decoding unit 20f executes the decoding process on the read image. After the first decoding process, the process proceeds to step SF3 to determine whether the reading was successful. If the reading is not successful, the process returns to step SF2 and the first decoding process is executed again. If the reading is not successful even after a predetermined number of times (time) have elapsed, a timeout occurs and the decoding process ends. On the other hand, if it is determined in step SF3 that the reading was successful, the decoding process ends.

[0170] Also, in parallel with the progress from step SF1 to step SF2, the process proceeds to step SF4. In step SF4, the extraction unit 20g extracts a code candidate region where a code is likely to exist from among the read images generated by the camera 5. Thereafter, the process proceeds to step SF5, where any one code candidate region is selected from among the plurality of code candidate regions extracted in step SF4, and the process proceeds to step SF6.

[0171] In step SF6, the super-resolution processing unit 20k performs super-resolution processing on the captured image generated by the camera 5 to generate an enlarged image. In this example, the dedicated core 25 executes the super-resolution processing, but the general-purpose cores 21 to 24 may also execute it. The generation of the enlarged image is executed in parallel with the first decoding process in step SF2. After generating the enlarged image in step SF6, the process proceeds to step SF7, where the decoding processing unit 20f performs decoding processing on the enlarged image generated in step SF6, i.e., the second decoding process. After the second decoding process, the process proceeds to step SF8 to determine whether the reading was successful. If the reading is not successful, the process returns to step SF5, selects another code candidate area, and then proceeds to steps SF6 and SF7 in sequence to execute the second decoding process again. If the reading is not successful even after a predetermined number of times (time) has elapsed, a timeout occurs and the decoding process ends. On the other hand, if it is determined in step SF8 that the reading was successful, the decoding process ends.

[0172] The case of proceeding to step SF2 and the case of proceeding to step SF4 can be distinguished by the PPC value. For example, a determination unit can be provided in the processor 20 to determine that when the PPC value exceeds a predetermined value, the process proceeds to step SF2, while when the PPC value is less than or equal to the predetermined value, the process proceeds to step SF4. Specifically, the determination unit can be configured such that when the PPC value is 2.0 or more, the process proceeds to step SF2, while when the PPC value is less than 2.0, the process proceeds to step SF4. Thereby, a wide range of PPC codes can be decoded. In particular, in the portable optical information reading device 1A, the distance to the code and the code size are often not constant, but even in such cases, the success rate of the decoding process can be increased in this example. Also, in the stationary optical information reading device 1, the installation freedom can be increased.

[0173] The optical information reading device 1 may be provided with a function to cancel steps SF4 to SF8. Steps SF4 to SF8 are steps for executing super-resolution processing and second decoding processing. However, since super-resolution processing particularly requires time, the processing can be speeded up by canceling steps SF4 to SF8. As a method for canceling steps SF4 to SF8, for example, there is a method by a user's operation. For example, as one of the parameters stored in the parameter set storage unit 40c, a selection parameter for whether or not to execute super-resolution processing may be included. By the user changing this selection parameter, it is possible to switch to either a mode for executing super-resolution processing or a mode for not executing it. The selection parameter may be, for example, a selection parameter for the "speed priority mode". When the "speed priority mode" is selected by the user, the operation is performed in a mode where super-resolution processing is not executed. On the other hand, when the "speed priority mode" is not selected, the operation is performed in a mode where super-resolution processing is executed. Incidentally, when it is determined based on the information obtained by tuning that the situation is such that super-resolution processing is unnecessary, super-resolution processing may be automatically canceled.

[0174] (Third example during operation including super-resolution processing) A third example of the case where super-resolution processing is executed during the operation of the optical information reading devices 1 and 1A will be described based on the flowchart shown in FIG. 23. The third example is an example applicable when reading a code using the handheld optical information reading device 1A. As described above, when reading a small code or a code at a long distance with the handheld optical information reading device 1A, the code is read relying on the aiming light. As shown in FIG. 5, since the aiming optical system usually has a different optical axis from the camera optical system, as shown in FIG. 6, the center of the aiming light 10d and the center of the field of view 5d of the camera 5 often deviate. Since the area for executing super-resolution processing is preferably small for reasons of processing time, in the third example, the area for executing super-resolution processing is limited by the positional relationship with the aiming light 10d.

[0175] After the start of the flowchart shown in FIG. 23, when a read image is input, the region extraction unit 20g detects the position of the aiming light 10d (shown in FIG. 8) at step SG1. In this step SG1, for example, a method of generating a read image in which the aiming light 10d as shown in FIG. 8 is captured by the camera 5 and detecting the position of the aiming light 10d by image processing, or the relationship between the distance between the optical information reading device 1A and the code and the position within the visual field 5d of the aiming light 10d is calculated in advance and stored in the storage device 30 or the like, and the distance between the optical information reading device 1A and the code is obtained using a well-known distance sensor or the focus information of the camera 5, etc., and the position within the visual field 5d of the aiming light 10d is calculated based on the obtained distance, etc. can be applied.

[0176] Thereafter, it proceeds to step SG2, and the region extraction unit 20g determines a code candidate region based on the position of the aiming light 10d. In this step SG2, as shown in FIG. 8, when the position of the aiming light 10d is detected, a region within a predetermined range including the center thereof is determined as the code candidate region, and the region is used as a partial image corresponding to the code candidate region.

[0177] Next, it proceeds to step SG3, and the filter processing unit 20c executes noise removal filter processing and contrast adjustment on the partial image corresponding to the code candidate region. Thereafter, it proceeds to step SG4, and the super-resolution processing unit 20k executes super-resolution processing on the partial image after the filter processing to generate an enlarged image. Steps SG5 to SG7 are the same as steps SE4 to SE6 of the flowchart shown in FIG. 19.

[0178] (Fourth example during operation including super-resolution processing) A fourth example of executing super-resolution processing during the operation of the optical information reading devices 1 and 1A will be described based on the flowchart shown in FIG. 24. The fourth example is an example applicable when reading a code using the handheld optical information reading device 1A, and includes control for switching between execution and non-execution of super-resolution processing according to the distance between the camera 5 and the code.

[0179] That is, as shown in FIG. 25, when it is desired to perform super-resolution processing, there are cases where the camera 5 and the code are separated and the pixel resolution is low at a long distance, and cases where a small code (small code) is read at a short distance. By measuring the distance between the camera 5 and the code and performing super-resolution processing only when the distance is short (distance d ≤ distance d1) or long (distance d ≥ distance d2), it is possible to avoid a decrease in processing speed due to performing super-resolution processing at a medium distance.

[0180] Steps SH1 to SH3 of the flowchart shown in FIG. 24 are the same as steps SF1 to SF3 of the flowchart shown in FIG. 22. While proceeding from step SH1 to step SH2, proceed to step SH4. In step SH4, the processor 20 determines whether the distance between the camera 5 and the code is at a medium distance. Specifically, as shown in FIG. 25, the distance d between the light receiving surface of the imaging element 5a of the camera 5 and the code is measured. The distance d can be obtained, for example, by using a distance sensor or the focus information of the camera 5. Then, if the distance d is less than or equal to the distance d1 (short distance) or the distance d is greater than or equal to the distance d2 (long distance), proceed to step SH5. On the other hand, if the distance d exceeds the distance d1 and is less than the distance d2 (medium distance), it is determined that the distance is one where super-resolution processing is not required, and the process ends. The distance d1 and the distance d2 can be set as the distances at which the PPC of the code included in the read image becomes a predetermined value or less.

[0181] Thereafter, proceed to step SH5. Steps SH5 to SH9 are the same as steps SF4 to SF8 of the flowchart shown in FIG. 22.

[0182] (Description of task sequence) FIG. 26 is a task sequence diagram when reading is successful in the first decoding process for decoding a read image. In this figure, a control task, a first decoding process task, a second decoding process task, and a super-resolution process task are shown. When the control task issues a read execution instruction 1 to the first decoding process task, the first decoding process starts. When reading is successful in the first decoding process, a read success notification 2 is sent to the control task. When receiving the read success notification 2, the control task issues a process end instruction 3 to the first decoding process task, and the first decoding process task sends a process end notification 4 to the control task.

[0183] Also, when the control task issues a read execution instruction 5 to the second decoding process task, the second decoding process task issues a super-resolution start instruction 6 to the super-resolution process task. When the super-resolution process is completed, the super-resolution process task sends a super-resolution completion notification 7 to the second decoding process task. Thereafter, the second decoding process task issues the next super-resolution start instruction 8 to the super-resolution process task. Thereafter, since the process end notification 4 is sent to the control task, the control task issues a process end instruction 9 to the second decoding process task. The second decoding process task issues a super-resolution end instruction 10 to the super-resolution process task, and the super-resolution process task sends a super-resolution end notification 11 to the second decoding process task. Next, the second decoding process task sends a process end notification 12 to the control task to end the reading. That is, the processor 20 is configured to end the super-resolution process midway even if the super-resolution process by the super-resolution processing unit 20j is not completed when decoding is successful in the first decoding process. Thereby, the next super-resolution process can be started earlier.

[0184] Figure 27 is a task sequence diagram when reading is successful in the second decoding process for decoding the enlarged image generated by the super-resolution process. The control task issues a read execution command 1 to the first decoding process task and a read execution command 2 to the second decoding process task. The second decoding process task issues a super-resolution start command 3 to the super-resolution process task. When the super-resolution process is completed, the super-resolution process task issues a super-resolution completion notification 4 to the second decoding process task. Thereafter, the second decoding process task issues the next super-resolution start command 5 to the super-resolution process task.

[0185] Upon receiving the super-resolution completion notification 4, the second decoding process task starts the second decoding process. When reading is successful in the second decoding process and the second decoding process task issues a read success notification 6 to the control task, the control task issues a process end command 7 to the second decoding process task. At this time, the control task also issues a process end command 8 to the first decoding process task.

[0186] Upon receiving the process end command 7, the second decoding process task issues a super-resolution end command 9 to the super-resolution process task, and the super-resolution process task issues a super-resolution end notification 10 to the second decoding process task. Also, the first decoding process task issues a process end notification 11 to the control task. Also, the second decoding process task issues a process end notification 12 to the control task. In this example, when decoding is successful in the second decoding process, the super-resolution process by the super-resolution processing unit 20k can be terminated.

[0187] Figure 28 is a task sequence diagram when reading is successful in the first decoding process and the second decoding process. The control task issues a read execution command 1 to the first decoding process task and a read execution command 2 to the second decoding process task. The second decoding process task issues a super-resolution start command 3 to the super-resolution process task. The first decoding process task starts the first decoding process. When reading is successful in the first decoding process and the first decoding process task issues a read success notification 4 to the control task, the control task issues a process end command 5 to the second decoding process task. The second decoding process task issues a process end notification 6 to the control task.

[0188] On the one hand, when the super-resolution processing task completes the super-resolution processing, it sends a super-resolution completion notification 7 to the second decoding processing task. After that, the second decoding processing task sends the next super-resolution start command 8 to the super-resolution processing task. The second decoding processing task starts the second decoding processing upon receiving the super-resolution completion notification 7. When the reading is successful in the second decoding processing and the second decoding processing task sends a reading success notification 9 to the control task, the control task sends a processing end command 10 to the second decoding processing task.

[0189] When the second decoding processing task receives the processing end command 10, it sends a super-resolution end command 11 to the super-resolution processing task, and the super-resolution processing task sends a super-resolution end notification 12 to the second decoding processing task. After receiving the super-resolution end notification 12, the second decoding processing task sends a processing end notification 13 to the control task. In this example, when the decoding is successful in both the first decoding processing and the second decoding processing, the super-resolution processing by the super-resolution processing unit 20k can be terminated.

[0190] FIG. 29 is a task sequence diagram when the reading is successful twice in the second decoding processing. The control task sends a reading execution command 1 to the first decoding processing task and a reading execution command 2 to the second decoding processing task. The second decoding processing task sends a super-resolution start command 3 to the super-resolution processing task. When the super-resolution processing task completes the super-resolution processing, it sends a super-resolution completion notification 4 to the second decoding processing task. After that, the second decoding processing task sends the next super-resolution start command 5 to the super-resolution processing task. The second decoding processing task starts the second decoding processing upon receiving the super-resolution completion notification 4. When the reading is successful in the second decoding processing and the second decoding processing task sends a reading success notification 6 to the control task.

[0191] After that, in the super-resolution processing task, the second super-resolution processing is completed, and the second super-resolution completion notification 7 is sent to the second decoding processing task. Upon receiving the second super-resolution completion notification 7, the second decoding processing task starts the second second decoding processing. Reading is also successful in the second second decoding processing, and the second decoding processing task sends a reading success notification 8 to the control task. During this period, since reading has not been successful in the first decoding processing task, no reading success notification is sent from the first decoding processing task.

[0192] The control task sends a processing end command 9 to the second decoding processing task. When the second decoding processing task receives the processing end command 9, it sends a super-resolution end command 10 to the super-resolution processing task, and the super-resolution processing task sends a super-resolution end notification 11 to the second decoding processing task. After receiving the super-resolution end notification 11, the second decoding processing task sends a processing end notification 12 to the control task. Also, when the control task sends a processing end command 13 to the first decoding processing task, the first decoding processing task sends a processing end notification 14 to the control task. In this example, when decoding is successful in both the first second decoding processing and the second second decoding processing, the super-resolution processing by the super-resolution processing unit 20k can be terminated.

[0193] (Operational effects of the embodiment) As described above, according to this embodiment, an enlarged image obtained by enlarging the PPC indicating the number of pixels of each module constituting the code included in the read image can be generated by a neural network. Thereby, the lower limit of the PCC that can be read can be lowered, so that, for example, decoding processing of a code photographed from a long distance or a code with a small module size becomes possible.

[0194] The above-described embodiment is merely illustrative in every respect and should not be construed in a limiting sense. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

Industrial applicability

[0195] As described above, the optical information reading device according to the present invention can be used, for example, when reading a code attached to a workpiece.

Explanation of Signs

[0196] 1 Stationary optical information reading device 1A Handheld optical information reading device 5, 5A Camera 10 Aimer light irradiation unit (light irradiation unit) 20 Processor 20f Decoding processing unit 20g Region extraction unit 20n PPC value acquisition unit 20k Super-resolution processing unit 21 - 24 First to fourth general-purpose cores 25 Dedicated core

Claims

1. In an optical information reading device for reading a code attached to a workpiece, A camera that captures an image of the code and generates a scanned image; a memory unit for storing the structure and parameters of a neural network that outputs an enlarged image of a PPC that indicates the number of pixels of each module that constitutes a code included in the read image generated by the camera; a processor including a dedicated core and a general-purpose core, for controlling the image capturing by the camera and for executing a decoding process on the enlarged image; The dedicated core reads out a structure and parameters of the neural network stored in the storage unit, configures a neural network using the structure and parameters, inputs the read image generated by the camera to the neural network, and generates an enlarged image by enlarging a PPC value; The general-purpose core is an optical information reader that executes imaging control of the camera.

2. 2. The optical information reading device according to claim 1, The general-purpose core is an optical information reader that outputs a result of the decoding process.

3. In an optical information reading device for reading a code attached to a workpiece, A camera that captures an image of the code and generates a scanned image; a memory unit for storing the structure and parameters of a neural network that outputs an enlarged image of a PPC that indicates the number of pixels of each module that constitutes a code included in the read image generated by the camera; a processor including a dedicated core and a general-purpose core, and performing a decoding process on the enlarged image; The dedicated core reads out a structure and parameters of the neural network stored in the storage unit, configures a neural network using the structure and parameters, inputs the read image generated by the camera to the neural network, and generates an enlarged image by enlarging a PPC value; The general-purpose core is an optical information reader that outputs a result of the decoding process.

4. 4. The optical information reading device according to claim 2, an optical information reading device configured so that the process of generating a first enlarged image by the dedicated core and the output of the result of the decoding process by the general-purpose core can be executed in parallel;

5. 5. The optical information reading device according to claim 4, the decoding process by the processor includes a first decoding process for decoding a read image generated by the camera, and a second decoding process for decoding a first enlarged image generated by the dedicated core; an optical information reading device configured so that the process of generating a second enlarged image by the dedicated core and the first and second decoding processes can be executed in parallel;

6. In an optical information reading device for reading a code attached to a workpiece, A camera that captures an image of the code and generates a scanned image; a memory unit for storing the structure and parameters of a neural network that outputs an enlarged image of a PPC that indicates the number of pixels of each module that constitutes a code included in the read image generated by the camera; a processor that inputs a scanned image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, generates an enlarged image by enlarging a PPC value of the input scanned image, and executes a decoding process on the enlarged image; a PPC value acquisition unit that acquires a scanned image generated by the camera when setting up the optical information reading device and acquires a PPC value of a code included in the acquired scanned image, The processor generates the enlarged image when the PPC value acquired by the PPC value acquisition unit is below a predetermined value, but does not generate the enlarged image when the PPC value acquired by the PPC value acquisition unit exceeds the predetermined value.

7. 7. The optical information reading device according to claim 6, An optical information reading device in which the processor generates an enlarged image and performs a decoding process on the enlarged image when the processor fails to decode a code contained in a read image generated by the camera when setting up the optical information reading device.

8. 7. The optical information reading device according to claim 1, a region extraction unit that searches for a code based on a feature amount for identifying the code from within the scanned image generated by the camera, and extracts a region including the searched code as a code candidate region; The processor inputs a partial image corresponding to the code candidate region extracted by the region extraction unit into the neural network to generate the enlarged image.

9. In an optical information reading device for reading a code attached to a workpiece, A camera that captures an image of the code and generates a scanned image; a memory unit for storing the structure and parameters of a neural network that outputs an enlarged image of a PPC that indicates the number of pixels of each module that constitutes a code included in the read image generated by the camera; a processor that inputs a scanned image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, generates an enlarged image by enlarging a PPC value of the input scanned image, and executes a decoding process on the enlarged image; a light irradiation unit for irradiating a visible light having a color different from that of the ambient light toward a photographing field of view of the camera to form a mark; a region extraction unit that identifies a portion of the read image generated by the camera that corresponds to the mark formed by the light irradiation unit and extracts the identified portion as a code candidate region, The processor inputs a partial image corresponding to the code candidate region extracted by the region extraction unit into the neural network to generate the enlarged image.

10. In the optical information reading device according to any one of claims 1 to 5, claim 8 depending on any one of claims 1 to 5, and claim 9, a PPC value acquisition unit for acquiring a PPC value of a code included in the scanned image generated by the camera; The processor generates the enlarged image when the PPC value acquired by the PPC value acquisition unit is below a predetermined value, but does not generate the enlarged image when the PPC value acquired by the PPC value acquisition unit exceeds the predetermined value.

11. 11. The optical information reading device according to claim 1, The processor includes a decoding processing unit that executes a first decoding process for decoding a read image generated by a camera and a second decoding process for decoding the enlarged image.

12. 12. The optical information reading device according to claim 11, The processor of the optical information reading device has a core that executes the first decoding process and the second decoding process in parallel in separate threads.

13. 12. The optical information reading device according to claim 11, The optical information reading device, wherein the decoding processing unit is configured with a multi-core capable of executing the first decoding process and the second decoding process using different cores.

14. 14. The optical information reading device according to claim 1, The optical information reading device further comprises a filter processing unit that applies a noise removal filter to the read image generated by the camera before inputting the read image to the neural network.

15. 15. The optical information reading device according to claim 1, The neural network is trained in advance using a low-resolution image as a defective image and an original high-resolution image of the defective image as a teacher image, and the memory unit is an optical information reading device that stores the structure and parameters of the neural network that has been trained in advance.

16. 16. The optical information reading device according to claim 1, The optical reader further comprises a housing that contains the processor and the camera.

Images