Optical information reader

Abstract

To preventing the load of inferring processing from causing a prolong of a processing time while increasing the accuracy of reading by inferring processing of a machine learning.SOLUTION: The optical information reading device has a processor including: an inferring processing unit for performing inferring processing of inputting a read image into a neural network and generating an ideal image for the read image; and a decoding processing unit of executing first decoding processing of decoding the read image and second decoding processing of decoding the ideal image generated by the inferring processing unit. The processor executes the inferring processing and the first decoding processing in parallel, and also executes the second decoding processing after the inferring processing is completed.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present invention relates to an optical information reading device that optically reads information. [Background technology]

[0002] In recent years, importance has been placed on traceability, which allows tracking of the distribution route of goods from the manufacturing stage to the consumption stage or disposal stage, and code readers for this purpose have become widespread.In addition to traceability, code readers are also used in various other fields.

[0003] Generally, a code reader is configured to use a camera to capture a barcode, two-dimensional code, or other code attached to a workpiece, extract the code contained in the resulting image using image processing, digitize it, and decode it to read the information.Since it is a device that reads information optically, it is also called an optical information reading device.

[0004] Known examples of this type of optical information reading device include one equipped with a machine learning device that learns a model structure that shows the relationship between a code image acquired by a robot's visual sensor and an ideal code image, as disclosed in Patent Document 1. Patent Document 1 describes that the results of learning by the machine learning device are applied to a code image acquired by a visual sensor during operation, thereby restoring the image to one suitable for reading. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6683666 Summary of the Invention [Problem to be solved by the invention]

[0006] However, image restoration using machine learning, i.e., inference processing of an ideal image, places a heavy computational load on the processor and takes a long time to process. Therefore, if the scanned image is not difficult to read, i.e., the scanned image is in good print condition, it is preferable to immediately perform decoding processing on the scanned image, taking into account processing time, without inputting it into a neural network and performing inference processing of an ideal image. On the other hand, it is not easy to determine in advance from the scanned image whether inference processing using machine learning is necessary, and performing decoding processing and inference processing using machine learning serially takes a long time to process.

[0007] The present invention has been made in consideration of these points, and its purpose is to improve reading accuracy through machine learning inference processing while preventing the processing time from increasing due to the load of the inference processing. [Means for solving the problem]

[0008] To achieve the above object, one aspect of the present disclosure can be based on an optical information reading device that reads a code attached to a workpiece. The optical information reading device includes a camera that photographs the code and generates a read image, a memory unit that stores the structure and parameters of a neural network for estimating an ideal image corresponding to the read image generated by the camera, an inference processing unit that inputs the read image generated by the camera to the neural network configured with the structure and parameters stored in the memory unit and performs an inference process to generate an ideal image corresponding to the read image, and a processor having a decoding processing unit that performs a first decoding process to decode the read image generated by the camera and a second decoding process to decode the ideal image generated by the inference processing unit. The processor is configured to perform the inference process by the inference processing unit and the first decoding process by the decoding processing unit in parallel, and to perform the second decoding process by the decoding processing unit after the inference process by the inference processing unit is completed.

[0009] According to this configuration, when the scanned image generated by the camera during operation of the optical information reading device is not suitable for reading the code, such as when the code is poorly printed, the scanned image generated by the camera is input to the inference processing unit and an ideal image is generated by performing inference processing. On the other hand, when the scanned image generated by the camera during operation of the optical information reading device is suitable for reading the code, such as when the code is well printed, the scanned image generated by the camera is not input to the inference processing unit and a first decoding processing is performed in the decoding processing unit. Since the inference processing and the first decoding processing are performed in parallel, while inference processing is being performed on a scanned image requiring inference processing, decoding processing can be performed on another scanned image that does not require inference processing, or while decoding processing is being performed on a scanned image that does not require inference processing, inference processing can be performed on another scanned image that requires inference processing. This reduces processing time. Note that the first decoding processing and the inference processing may be performed on the same scanned image.

[0010] Furthermore, after the inference processing by the inference processing unit is completed, the second decoding processing is executed to decode the ideal image, thereby improving the reading accuracy.

[0011] In another aspect of the present disclosure, the decoding processing unit may have a core that executes the first decoding process and the second decoding process in parallel using different threads.

[0012] In another aspect of the present disclosure, the decoding processing unit may be configured with multiple cores capable of executing the first decoding process and the second decoding process using different cores.

[0013] According to this configuration, the core that executes the first decoding process and the core that executes the second decoding process are different cores, so the first decoding process and the second decoding process can be executed in parallel, thereby reducing processing time.

[0014] In another aspect of the present disclosure, the decoding processing unit has a general-purpose core that executes an extraction process to extract a code candidate area that is likely to contain a code from the scanned image generated by the camera, and the first decoding process to decode a partial image extracted from the scanned image by the extraction process. The inference processing unit has a dedicated inference processing core that inputs the partial image extracted from the scanned image by the extraction process and executes an inference process to generate an ideal image corresponding to the scanned image.

[0015] According to this configuration, the process of extracting the code candidate area and the first decoding process of decoding the partial image extracted by the extraction process can be executed by a general-purpose core, and the inference process can be executed by a dedicated inference processing core that is specialized for inference processing and is separate from the general-purpose core, thereby speeding up the inference process.

[0016] In another aspect of the present disclosure, the general-purpose core executes a second decoding process that decodes an ideal image generated by the inference processing performed by the dedicated inference processing core.

[0017] In another aspect of the present disclosure, the general-purpose core has a first core that performs the extraction process, a second core that performs the first decoding process, and a third core that performs the second decoding process.

[0018] With this configuration, the extraction process, the first decoding process, and the second decoding process can all be executed by different cores. In particular, the second and third cores can be specialized for the decoding process, further speeding up the process.

[0019] In another aspect of the present disclosure, when an ideal image is generated by inference processing by the dedicated inference processing core, the general-purpose core executes the second decoding process to decode the ideal image in priority to the first decoding process.

[0020] In other words, when an ideal image is generated by inference processing, the first decoding process for the scanned image generated by the camera takes a long time, and it is considered that the image is difficult to read. Therefore, when an ideal image is generated, decoding the ideal image takes priority over the first decoding process for the scanned image generated by the camera, thereby resulting in a highly accurate read result being obtained in a short time.

[0021] In another aspect of the present disclosure, when a plurality of partial images are extracted as a result of the extraction process by the general-purpose core, the dedicated inference processing core can execute the inference process on the plurality of partial images in a predetermined order. Also, when a plurality of partial images are extracted as a result of the extraction process by the general-purpose core, the dedicated inference processing core can execute the first decoding process on the plurality of partial images in a predetermined order.

[0022] That is, when a workpiece is photographed and one scanned image is generated, there may be multiple code candidate areas in that single scanned image. In this case, multiple partial images are extracted. For the multiple partial images, the inference process can be performed, for example, in descending order of priority. Also, for the multiple partial images, the first decoding process can be performed, for example, in descending order of priority. The priority may be based on a condition for determining a predetermined order, and the process may be performed starting with the partial image with the highest probability of being a code.

[0023] In another aspect of the present disclosure, the processor is configured to terminate the inference processing by the inference processing unit if the decoding is successful in the first decoding processing.

[0024] That is, the first decoding process is a decoding process for a scanned image that has not undergone inference processing. If the decoding process for this scanned image is successful, it means that the scanned image is not difficult to read, and inference processing is unnecessary. In this case, by terminating the inference processing even if it is in progress, it becomes possible to start inference processing for another scanned image that is input next at an early stage.

[0025] In another aspect of the present disclosure, the general-purpose core can execute a first extraction process that extracts the code candidate region under a first predetermined condition, a second extraction process that extracts the code candidate region under a second predetermined condition that enables extraction with higher accuracy than the first predetermined condition, and the first decoding process that decodes a partial image extracted from the read image by the first extraction process. Also, the dedicated inference processing core can execute an inference process that inputs the partial image extracted from the read image by the second extraction process and generates an ideal image corresponding to the read image.

[0026] With this configuration, the general-purpose core can perform a first extraction process with relatively low accuracy and a second extraction process with relatively high accuracy. The first extraction process is less accurate, but allows for faster extraction. The partial image extracted from the scanned image by the first extraction process is decoded by the first decoding process, and if the decoding is successful, the scanned result can be output in a short time.

[0027] On the other hand, a read image that fails to be decoded in the first decoding process may be difficult to read and difficult to extract in the first extraction process. In this case, the partial image extracted in the second extraction process, which allows for more accurate extraction, is subjected to inference processing by a core dedicated to inference processing to generate an ideal image. By performing the second decoding process on the generated ideal image, reading accuracy is improved.

[0028] In other words, the first decoding process can quickly output the reading results of a read image that is easy to read, and the second decoding process can read even read images that are likely to fail to be decoded in the first decoding process, thereby reducing the frequency of decoding failures.

[0029] In another aspect of the present disclosure, the optical information reading device is provided with a memory in which a partial image extracted from the read image by the extraction process is stored and which is accessible by both the general-purpose core and the dedicated inference processing core.

[0030] This configuration allows the general-purpose core and the dedicated inference processing core to share the extraction results, thereby speeding up processing. The memory is configured with RAM, etc., and the general-purpose core and the dedicated inference processing core can be configured to access the same RAM.

[0031] In another aspect of the present disclosure, the general-purpose core and the dedicated inference processing core may be mounted on the same board, in which case the dedicated inference processing core may be configured as an IC, FPGA, or the like dedicated to inference processing using a neural network.

[0032] In another aspect of the present disclosure, the image restoration system further includes an image restoration unit that inputs a scanned image to a neural network and performs an inference process to generate a restored image by restoring the scanned image; a first decoding processing unit that performs a first decoding process on the scanned image generated by the camera; and a second decoding processing unit that performs a second decoding process on the restored image generated by the image restoration unit. The second decoding processing unit extracts a code candidate area from the scanned image and sends a trigger signal to the image restoration unit to execute the inference process. Upon receiving the trigger signal from the second decoding processing unit, the image restoration unit inputs a partial image corresponding to the code candidate area to the neural network and executes an inference process to generate a restored image by restoring the partial image. The second decoding processing unit can determine grid positions indicating the positions of each cell of the code based on the restored image and decode the restored image based on the determined grid positions. In other words, the first decoding processing unit decodes a normal read image that has not been subjected to inference processing, the image restoration unit generates a restored image through inference processing, and the second decoding processing unit decodes the restored image restored through inference processing, thereby constructing processing circuits suited to each and increasing the processing speed.In addition, the image restoration unit only needs to restore a partial image corresponding to the code candidate area, which enables further increase in processing speed.

[0033] In another aspect of the present disclosure, the inference processing by the image restoration unit and the first decoding processing by the first decoding processing unit are performed in parallel, thereby increasing the processing speed. Furthermore, at least a portion of the period during which the second decoding processing unit is performing the second decoding processing includes a pause period during which the inference processing by the image restoration unit is not performed, thereby reducing the load on the image restoration unit and suppressing heat generation. In this case, the second decoding processing unit may be configured with multiple cores.

[0034] In another aspect of the present disclosure, the first decoding processing unit can execute the first decoding process in a shorter time than the time required for the second decoding process. Furthermore, the setting unit can switch whether or not to execute the second decoding process, i.e., whether or not to execute decoding of the restored image. The second decoding process, which executes decoding of the restored image, takes longer than the first decoding process, which executes decoding of a normal scanned image. Given this premise, when the second decoding process is set to be executed, a decoding timeout period (decoding time limit) longer than the time required for executing the second decoding process can be set. On the other hand, when the second decoding process is set not to be executed, a decoding timeout period shorter than the time required for executing the second decoding process can be set. Therefore, if the decoding of the restored image exceeds the decoding timeout period, the process can quickly transition to decoding of the next captured image.

[0035] A decoding processing unit according to another aspect of the present disclosure can perform a second decoding process on a restored image in parallel with a first decoding process on a scanned image generated by a camera, thereby increasing the processing speed even when scanned images and restored images are mixed. When the setting unit sets the decoding processing unit not to perform the second decoding process, the decoding processing unit can allocate processing resources used for the second decoding process to the first decoding process. Processing resources include, for example, memory usage areas and cores constituting a multi-core CPU. When a memory area or core is previously set as a processing resource used for the second decoding process, setting the second decoding process not to be performed allows the memory area or core to be used for the first decoding process, thereby further increasing the speed of the first decoding process.

[0036] A decoding processing unit according to another aspect of the present disclosure extracts a code candidate area from a scanned image that is likely to contain a code. An image restoration unit inputs a partial image corresponding to the code candidate area into a neural network and generates a restored image by restoring the partial image. A decoding processing unit determines grid positions based on the restored image generated by the image restoration unit and decodes the restored image based on the determined grid positions. Since generating a restored image often takes time, the time required to generate the restored image can be reduced by extracting a code candidate area based on, for example, feature quantities that indicate code resemblance before generating the restored image. Furthermore, since the grid positions can be determined with high accuracy based on the restored image, reading performance is improved.

[0037] In another aspect of the present disclosure, a tuning process is performed to determine optimal imaging conditions and decoding conditions and to set the size of the code to be read. The decoding processing unit can determine the size of an extracted image containing a code candidate area from the scanned image based on the code size set in the tuning process. By enlarging or reducing the extracted image to a predetermined size and inputting it into a neural network, the PPC of the code image input to the neural network can be kept within a predetermined range, thereby stably achieving the image restoration effect of the neural network. Furthermore, although the code size to be read may vary, the size of the code ultimately input to the neural network can be fixed, thereby maintaining a constant balance between high processing speed and ease of decoding.

[0038] A camera according to another aspect of the present disclosure receives light that passes through a first polarizing plate and is reflected from the code via a second polarizing plate, generating a read image with lower contrast than when the light is not passed through the first and second polarizing plates. By using a polarizing plate, the specular reflection component of the workpiece is removed, resulting in a read image with reduced influence of the specular reflection component. While polarizing plates are suitable for images with a high specular reflection component, such as when capturing an image of a metal workpiece, reduced light intensity can cause the read image to become dark and the contrast to decrease. In such cases, reading performance can be improved by converting the image into a high-contrast restored image using neural network inference processing.

[0039] In another aspect of the present disclosure, the tuning execution unit can set a plurality of imaging conditions and code conditions including first imaging conditions and code conditions and second imaging conditions and code conditions. The decoding processing unit can input a read image generated under the first imaging conditions and code conditions to a neural network to generate a repaired image, and if decoding of the repaired image fails, input a read image generated under second imaging conditions and code conditions different from the first imaging conditions and code conditions to the neural network to generate a repaired image, and decode the repaired image. [Effects of the Invention]

[0040] As described above, the inference processing by the logic processing unit and the decoding processing by the decoding processing unit can be performed in parallel, and the decoding processing can be performed on the ideal image after the inference processing is completed. This makes it possible to improve reading accuracy through the inference processing while preventing the processing time from becoming longer due to the load of the inference processing. [Brief explanation of the drawings]

[0041] [Figure 1] FIG. 10 is a diagram illustrating the stationary optical information reader during operation. [Figure 2] FIG. 1 is a perspective view of a stationary optical information reading device. [Figure 3]FIG. 1 is a block diagram of an optical information reader. [Figure 4] FIG. 1 is a perspective view of a handheld optical information reading device. [Figure 5] FIG. 2 is a diagram illustrating each unit configured by a processor. [Figure 6] FIG. 1 is a conceptual diagram of a neural network. [Figure 7] 1 is a flowchart showing an example of a basic procedure for learning a neural network. [Figure 8] A pair of defective and ideal images is shown, where (A) is an example of the defective image and (B) is an example of the ideal image. [Figure 9] FIG. 1 is a conceptual diagram of a convolutional neural network used for image conversion. [Figure 10] 10 is a flowchart showing an example of the procedure of a tuning process performed when setting up an optical information reader. [Figure 11] 10 is a flowchart illustrating an example of a decoding process procedure before a reduction ratio and an enlargement ratio are determined. [Figure 12] 10 is a flowchart illustrating an example of a decoding process procedure after a reduction ratio and an enlargement ratio are determined. [Figure 13] 1A shows an example of a scanned image generated by a camera, and FIG. 1B shows an example of an image after image processing filtering. [Figure 14] 10A shows an example of an image after reduction processing, and FIG. 10B shows an example of an image after code region extraction. [Figure 15] FIG. 10 is a diagram showing an example in which an inference process is performed on a scanned image by a neural network. [Figure 16] 10 is a flowchart showing a first example of a case where decoding processing and inference processing for a read image are executed in parallel. [Figure 17A] FIG. 10 is a sequence diagram showing sequence example 1 of the first example. [Figure 17B] FIG. 10 is a sequence diagram showing a sequence example 2 of the first example. [Figure 17C] FIG. 10 is a sequence diagram showing a sequence example 3 of the first example. [Figure 18] 10 is a flowchart showing a second example of a case where decoding processing and inference processing for a read image are executed in parallel. [Figure 19] FIG. 10 is a sequence diagram illustrating an example of a sequence of a second example. [Figure 20] 10 is a flowchart showing a third example of a case where decoding processing and inference processing for a read image are executed in parallel. [Figure 21] FIG. 10 is a sequence diagram illustrating an example of a sequence of a third example. [Figure 22] FIG. 10 is a task sequence diagram when reading is successful in the first decoding process. [Figure 23] FIG. 10 is a task sequence diagram when reading is successful in the second decoding process. [Figure 24] FIG. 10 is a task sequence diagram when reading is successful in the first decoding process and the second decoding process. [Figure 25] FIG. 10 is a task sequence diagram when reading is successful twice in the second decoding process. [Figure 26] FIG. 10 is a diagram showing a reading sequence during continuous shooting. [Figure 27] FIG. 10 is a sequence diagram of a decoding process during continuous shooting. [Figure 28] FIG. 10 is a diagram illustrating an example of a decoding process procedure according to another embodiment. [Figure 29] 4 is a time chart of the decoding process by the first decoding processing unit and the second decoding processing unit. [Figure 30] 10 is a time chart of a decoding process by a decoding processing unit. [Figure 31] 10 is a flowchart showing an example of a first decoding process and a second decoding process. [Figure 32] 10A to 10C are diagrams illustrating example images of each process. [Figure 33] 10A and 10B are diagrams illustrating a case where a plurality of neural networks are used to deal with black and white inversion of a code. [Figure 34]FIG. 10 is a diagram illustrating a case where one neural network is used to handle black and white inversion of a code. [Figure 35] 10 is a graph showing the relationship between contrast and matching level. [Figure 36] 10 is a flowchart illustrating an example of a process for calculating a matching level. [Figure 37] FIG. 4 is a diagram illustrating an example of a user interface displayed on a display unit. [Figure 38] FIG. 10 is a diagram showing an example in which an imaging element with an AI chip is used. DETAILED DESCRIPTION OF THE INVENTION

[0042] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present invention, its applications, or its uses.

[0043] (Stationary optical information reader) FIG. 1 is a schematic diagram illustrating a stationary optical information reader 1 according to an embodiment of the present invention during operation. In this example, multiple workpieces W are placed on the upper surface of a conveyor belt B and transported in the direction of arrow Y in FIG. 1 . The optical information reader 1 according to the embodiment is installed above and away from the workpieces W. The optical information reader 1 is a code reader configured to photograph a code attached to the workpiece W and decode the code contained in the photographed image to read information. In the example shown in FIG. 1 , the optical information reader 1 is a stationary type. When this stationary optical information reader 1 is in operation, the optical information reader 1 is fixed to a bracket or the like (not shown) to prevent movement. The stationary optical information reader 1 may also be used while being held by a robot (not shown). Alternatively, the optical information reader 1 may be configured to read the code on a stationary workpiece W. The stationary optical information reader 1 is in operation when it is performing an operation of reading the codes of the works W conveyed by the conveyor belt B in order.

[0044] In addition, a code is attached to the outer surface of each workpiece W. The code includes both barcodes and two-dimensional codes. Examples of two-dimensional codes include QR Code (registered trademark), Micro QR Code, Data Matrix (Data code), Veri Code, Aztec Code, PDF417, and Maxi Code. Two-dimensional codes are available in stack and matrix types, and the present invention can be applied to any two-dimensional code. The code may be attached by printing or engraving directly onto the workpiece W, or by printing on a label and then attaching it to the workpiece W; the means and method are not important.

[0045] The optical information reader 1 is connected to a computer 100 and a programmable logic controller (PLC) 101 by wired connection via signal lines 100a and 101a, respectively. However, this is not limiting, and the optical information reader 1, the computer 100, and the PLC 101 may each have a built-in communication module, and the optical information reader 1 may be connected to the computer 100 and the PLC 101 wirelessly. The PLC 101 is a control device for sequentially controlling the conveyor belt B and the optical information reader 1, and a general-purpose PLC may be used. The computer 100 may be a general-purpose or dedicated electronic computer, a portable terminal, or the like.

[0046] Furthermore, during operation, the optical information reading device 1 receives a read start trigger signal from the PLC 101 via signal line 101a, which signal specifies the timing for starting code reading. The optical information reading device 1 then captures and decodes the code based on this read start trigger signal. The decoded result is then transmitted to the PLC 101 via signal line 101a. Thus, during operation of the optical information reading device 1, the read start trigger signal is repeatedly input and the decoded result is repeatedly output via signal line 101a between the optical information reading device 1 and an external control device such as the PLC 101. Note that the input of the read start trigger signal and the output of the decoded result may be performed via signal line 101a between the optical information reading device 1 and the PLC 101, as described above, or via another signal line (not shown). For example, a sensor for detecting the arrival of a workpiece W may be directly connected to the optical information reading device 1, and the read start trigger signal may be input from the sensor to the optical information reading device 1.

[0047] 2, the optical information reader 1 is provided with a box-shaped housing 2, a polarizing filter attachment 3, an illumination unit 4, a camera 5, a display unit 6, a power connector 7, and a signal line connector 8. Furthermore, the housing 2 is provided with an indicator 9, an aimer light emitting unit (light emitting unit) 10, and operation buttons 11 and 12, and the indicator 9, the aimer light emitting unit 10, and the operation buttons 11 and 12 are also components of the optical information reader 1.

[0048] The housing 2 has a shape that is long in a predetermined direction, but the shape of the housing 2 is not limited to the shape shown in the figure. A polarizing filter attachment 3 is detachably attached to the outer surface of the front side of the housing 2. Inside this housing 2, an illumination unit 4, a camera 5, an aimer light irradiation unit 10, a processor 20, a storage device 30, a ROM 40, a RAM 41, etc. are housed. The processor 20, the storage device 30, the ROM 40, and the RAM 41 are also components of the optical information reading device 1.

[0049] An illumination unit 4 is provided on the front side of the housing 2. The illumination unit 4 is a part for illuminating at least the code of the workpiece W by emitting light toward the front of the optical information reading device 1. As also shown in FIG. 3, the illumination unit 4 includes a first illumination unit 4a consisting of a plurality of light-emitting diodes (LEDs), a second illumination unit 4b consisting of a plurality of light-emitting diodes, and an illumination drive unit 4c consisting of an LED driver or the like that drives the first illumination unit 4a and the second illumination unit 4b. The first illumination unit 4a and the second illumination unit 4b are driven individually by the illumination drive unit 4c so that they can be turned on and off separately. The illumination drive unit 4c is connected to the processor 20, and the illumination drive unit 4c is controlled by the processor 20. Note that one of the first illumination unit 4a and the second illumination unit 4b may be omitted.

[0050] As shown in FIG. 2, a camera 5 is provided in the center of the front side of the housing 2. The optical axis direction of the camera 5 is approximately the same as the direction of light irradiation by the illumination unit 4. The camera 5 is a part that photographs the code and generates a read image. As shown in FIG. 3, the camera 5 is equipped with an imaging element 5a that receives reflected light from the code attached to the workpiece W and illuminated by the illumination unit 4, an optical system 5b having a lens etc., and an AF module (autofocus module) 5c. Light reflected from the part of the workpiece W where the code is attached is incident on the optical system 5b, and the incident light is emitted toward the imaging element 5a and forms an image on the imaging surface of the imaging element 5a.

[0051] The imaging element 5a is an image sensor consisting of light-receiving elements such as a CCD (charge-coupled device) or a CMOS (complementary metal oxide semiconductor) that converts the image of the code obtained through the optical system 5b into an electrical signal. The imaging element 5a is connected to the processor 20, and the electrical signal converted by the imaging element 5a is input to the processor 20 as data of the read image. The AF module 5c is a mechanism that adjusts the focus by changing the position and refractive index of the focusing lens that constitutes the optical system 5b. The AF module 5c is also connected to the processor 20 and is controlled by the processor 20.

[0052] As shown in FIG. 2, a display unit 6 is provided on the side of the housing 2. The display unit 6 is, for example, an organic electroluminescence (EL) display or a liquid crystal display. The display unit 6 is connected to the processor 20 and can display, for example, the code captured by the imaging unit 5, the character string resulting from decoding the code, the read success rate, the matching level, etc. The read success rate is the average read success rate when a reading process is performed multiple times. The matching level is the reading margin that indicates the ease of reading a successfully decoded code. This can be determined from the number of error corrections that occurred during decoding, and can be expressed, for example, as a numerical value. The fewer the error corrections, the higher the matching level (reading margin), and conversely, the more the error corrections, the lower the matching level (reading margin).

[0053] A power cable (not shown) for supplying power from an external source to the optical information reader 1 is connected to the power connector 7. Signal lines 100a, 101a, etc. for communicating with the computer 100 and the PLC 101 are connected to the signal line connector 8. The signal line connector 8 can be configured, for example, by an Ethernet connector, a serial communication connector such as RS232C, a USB connector, etc.

[0054] An indicator 9 is provided on the housing 2. The indicator 9 is connected to the processor 20 and can be configured with a light-emitting element such as a light-emitting diode. The operating state of the optical information reader 1 can be notified to the outside by the lighting state of the indicator 9.

[0055] A pair of aimer light emitting units 10 are provided on the front side of the housing 2, sandwiching the camera 5. As shown in FIG. 3, the aimer light emitting unit 10 includes an aimer 10a formed of a light-emitting diode or the like, and an aimer driving unit 10b that drives the aimer 10a. The aimer 10a emits light (aimer light) toward the front of the optical information reading device 1 to indicate the shooting range and center of the field of view of the camera 5, a guide for the optical axis of the illumination unit 4, and the like. Specifically, the aimer 10a emits visible light of a color different from the ambient light (e.g., red or green) toward the shooting field of view of the camera 5, and forms a mark visible to the naked eye on the surface illuminated by the visible light. The mark may be various shapes, symbols, letters, or the like. The user can also install the optical information reading device 1 by referring to the light emitted from the aimer 10a.

[0056] As shown in FIG. 2, operation buttons 11 and 12 are provided on the side of the housing 2 to be used when setting up the optical information reader 1, etc. The operation buttons 11 and 12 include, for example, a select button and an enter button. In addition to the operation buttons 11 and 12, for example, a touch panel type operation means may be provided. The operation buttons 11 and 12 are connected to the processor 20, and the processor 20 is capable of detecting the operation states of the operation buttons 11 and 12. By operating the operation buttons 11 and 12, it is possible to select one option from multiple options displayed on the display unit 6 and to confirm the selection.

[0057] (Handheld optical information reader) In the above example, the optical information reader 1 is a stationary type, but the present invention can be applied to devices other than the stationary type optical information reader 1. Fig. 4 shows a handheld type optical information reader 1A, and the present invention can also be applied to a handheld type optical information reader 1A such as that shown in this figure.

[0058] The housing 2A of the handheld optical information reader 1A is elongated in the vertical direction. The orientation of the optical information reader 1A during use is not limited to the orientation shown in the figure, and the optical information reader 1A can be used in various orientations, but for the sake of convenience of explanation, the vertical direction of the optical information reader 1A is specified.

[0059] A display unit 6A is provided on the upper part of the housing 2A. The display unit 6A is configured in the same manner as the display unit 6 of the stationary optical information reader 1. The lower part of the housing 2A is a gripping unit 2B that the user holds during operation. The gripping unit 2B is a part that can be held in one hand by an average adult, and the shape and size can be freely set. By holding this gripping unit 2B, the optical information reader 1A can be carried around. In other words, this optical information reader 1A is a portable terminal device, and can also be called, for example, a handy terminal.

[0060] Like the stationary optical information reader 1, the handheld housing 2A also contains an illumination unit, a camera, an aimer light emitting unit, a processor, a memory unit, ROM, RAM, etc. (not shown). The optical axes of the illumination unit, the camera, and the aimer light emitting unit are directed diagonally upward from near the top end of the housing 2A. The handheld housing 2A is also provided with a buzzer (not shown).

[0061] A plurality of operation buttons 11A and a trigger key 11B are provided on or near the gripping portion 2B. The operation button 11A is similar to the operation button 11 of the stationary optical information reading device 1. When the user points the tip (top end) of the optical reading device 1A toward the workpiece W and presses the trigger key 11B, an aimer light is emitted from the tip of the optical reading device 1A. The user adjusts the orientation of the optical reading device 1A while visually observing the aimer light reflected on the surface of the workpiece W, and aligns the aimer light with the code to be read, and the code is automatically read and decoded. When reading is complete, a buzzer emits a completion notification sound.

[0062] An example of the use of the handheld optical information reading device 1A is in the picking work in a logistics warehouse. For example, when an ordered product is to be shipped from the logistics warehouse, the required product is picked from a product shelf in the warehouse. This picking work is performed by a user holding an order slip with a code written on it, who then goes to the product shelf and compares the code on the order slip with the code attached to the product or the product shelf. In this case, the handheld optical information reading device 1A alternately reads the code on the order slip and the code attached to the product or the product shelf.

[0063] (Processor configuration) The following description is common to both the stationary optical information reader 1 and the handheld optical information reader 1A, and is applicable to either device 1 or 1A unless otherwise specified. As shown in FIG. 3, the processor 20 is configured as a multi-core processor having multiple cores, which are central processing units. Specifically, the processor 20 includes a first general-purpose core 21, a second general-purpose core 22, a third general-purpose core 23, a fourth general-purpose core 24, and a dedicated core 25 dedicated to inference processing using a neural network. The first to fourth general-purpose cores 21 to 24 and the dedicated core 25 are so-called System-on-a-chip (SoC, SOC) and are mounted on the same board. The first to fourth general-purpose cores 21 to 24 and the dedicated core 25 do not necessarily have to be SoCs, in which case they do not have to be mounted on the same board, and this case is also within the scope of the present invention. In this embodiment, a case where the number of general-purpose cores is four will be described, but this is not limited to this, and the number of general-purpose cores may be one, or any number greater than or equal to two (for example, six cores, eight cores, etc.).

[0064] The first to fourth general-purpose cores 21 to 24 and the dedicated core 25 are connected to a RAM 41 as the same memory, and all of the first to fourth general-purpose cores 21 to 24 and the dedicated core 25 can access the same RAM 41. In addition, the processor 20 is connected to a ROM 40 as the same memory, and all of the first to fourth general-purpose cores 21 to 24 and the dedicated core 25 can also access the same ROM 40.

[0065] The first to fourth general-purpose cores 21 to 24 are so-called general-purpose processors, and are parts that execute, for example, AF control, lighting control, camera control, extraction processing for extracting code candidate areas, decoding processing for scanned images, various filter processing for scanned images, etc. Specific examples of processing by the first to fourth general-purpose cores 21 to 24 will be described later.

[0066] On the other hand, the dedicated core 25 is a dedicated inference processing core for executing inference processing to generate an ideal image corresponding to the scanned image using a neural network, and is specialized for ultra-high-speed execution of the product-sum operations required for neural network processing. The dedicated core 25 includes, for example, an IC or an FPGA. Note that by applying the learning results of the neural network to perform inference processing, the scanned image can be restored to an image suitable for decoding, so generating an ideal image through inference processing is also called restoration of the scanned image. In this case, the dedicated core 25 is the part that attempts to restore the scanned image using the neural network.

[0067] 5, the processor 20 includes an AF control unit 20a, an imaging control unit 20b, a filter processing unit 20c, a tuning execution unit 20d, a decoding processing unit 20f, an extraction unit 20g, a reduction unit 20h, an enlargement unit 20i, and an inference processing unit 20j. The AF control unit 20a, the imaging control unit 20b, the filter processing unit 20c, the tuning execution unit 20d, the decoding processing unit 20f, the extraction unit 20g, the reduction unit 20h, and the enlargement unit 20i are configured by the arithmetic processing of the first to fourth general-purpose cores 21 to 24. On the other hand, the inference processing unit 20j is configured by the dedicated core 25, and is also referred to as an image restoration unit because it is a unit that executes inference processing to generate a restored image by restoring a scanned image.

[0068] (AF control unit configuration) 3, and is configured to be able to focus the optical system 5b by conventionally known contrast AF or phase difference AF. The AF control unit 20a may be configured by any of the first to fourth general-purpose cores 21 to 24 that become the decode processing unit 20f and the extraction unit 20g, or by any core other than the core that becomes the decode processing unit 20f and the extraction unit 20g.

[0069] (Configuration of imaging control unit) The imaging control unit 20b is a unit that adjusts the gain of the camera 5, controls the light intensity of the illumination unit 4, and controls the exposure time (shutter speed) of the image sensor 5a. Here, the gain of the camera 5 refers to the amplification factor (also called magnification) used when amplifying the brightness of the image output from the image sensor 5a through digital image processing. The light intensity of the illumination unit 4 can be changed by separately controlling the first illumination unit 4a and the second illumination unit 4b. The gain, light intensity of the illumination unit 4, and exposure time are imaging conditions of the camera 5. The imaging control unit 20b may be configured by one of the first to fourth general-purpose cores 21 to 24 that serves as the decode processing unit 20f and the extraction unit 20g, or by a core other than the core that serves as the decode processing unit 20f and the extraction unit 20g. The AF control unit 20a and the imaging control unit 20b may be configured by the same core or different cores.

[0070] (Configuration of the filter processing unit) The filter processing unit 20c is a part that executes image processing filtering on the scanned image, and may be configured with a core that becomes the decode processing unit 20f and the extraction unit 20g, or may be configured with a core other than the core that becomes the decode processing unit 20f and the extraction unit 20g. The core that configures the filter processing unit 20c may be a DSP core.

[0071] The filter processing unit 20c executes a noise removal filter that removes noise contained in the image generated by the camera 5, a contrast correction filter that corrects contrast, an averaging filter, etc. The image processing filters executed by the filter processing unit 20c are not limited to a noise removal filter, a contrast correction filter, and an averaging filter, and may include other image processing filters.

[0072] The filter processing unit 20c is configured to apply image processing filtering to the read image before execution of the inference processing described below. Also, the filter processing unit 20c is configured to apply image processing filtering to the read image before enlargement or reduction described below. Note that the filter processing unit 20c may be configured to apply image processing filtering to the read image after execution of the inference processing, or may be configured to apply image processing filtering to the read image after enlargement or reduction.

[0073] (Configuration of tuning execution unit) The tuning execution unit 20d shown in FIG. 5 repeatedly captures and decodes the code while varying the imaging conditions of the camera 5 and the decoding conditions of the decoding process. Based on a matching level indicating the ease of code reading calculated under each imaging and decoding condition, the tuning execution unit 20d determines optimal imaging and decoding conditions and sets the size of the code to be read. Specifically, when configuring the optical information reading device 1, 1A, the tuning execution unit 20d sets various conditions (tuning parameters) to achieve optimal decoding conditions by changing imaging conditions such as the gain of the camera 5, the light intensity and exposure time of the illumination unit 4, and the image processing conditions in the filter processing unit 20c. The image processing conditions in the filter processing unit 20c include the coefficients (strength of the image processing filter), switching between image processing filters if multiple image processing filters are used, and combining different types of image processing filters. Appropriate imaging and image processing conditions vary depending on factors such as the effect of external light on the workpiece W during transport and the color and material of the surface on which the code is attached. Therefore, the tuning execution unit 20d searches for more appropriate imaging conditions and image processing conditions, and sets the processing to be performed by the AF control unit 20a, the imaging control unit 20b, and the filter processing unit 20c.

[0074] The tuning execution unit 20d is configured to be able to acquire the size of a code included in a scanned image generated by the camera 5. To acquire the code size, the tuning execution unit 20d first searches for the code based on a feature that indicates its likelihood of being a code. Next, the tuning execution unit 20d acquires the PPC, code type, code size, etc. of the searched code. The PPC, code type, code size, etc. are included in the code parameters or code conditions, and therefore the tuning execution unit 20d is configured to be able to acquire the code parameters or code conditions of the searched code.

[0075] A two-dimensional code is composed of multiple white and black modules arranged randomly. For example, in QR Code Model 2, the number of modules ranges from 21 x 21 to 177 x 177. In the case of a stationary optical information reader 1, the number of modules and PPC are limited by reading the code to be read in advance, and the code size (pixel size) is limited to the number of modules x PPC.

[0076] When focusing on one of the modules that make up a code, the PPC is a code parameter that indicates how many pixels (picture elements) that module is made up of. After identifying one module, the tuning execution unit 20d can obtain the PPC by counting the number of pixels that make up that module.

[0077] The code type refers to the type of code, such as QR code, data matrix code, VeriCode, etc. Since each code has its own characteristics, tuning execution unit 20d can distinguish the code type, for example, based on the presence or absence of a finder pattern.

[0078] The code size can be calculated from the number of modules arranged vertically or horizontally in the code and the PPC. The tuning execution unit 20d calculates the number of modules arranged vertically or horizontally in the code, calculates the PPC, and obtains the code size by multiplying the number of modules arranged vertically or horizontally by the PPC.

[0079] (Configuration of the decoding processing unit) The decoding processing unit 20f decodes the black and white binary data. A table showing the correspondence between the coded data can be used for decoding. Furthermore, the decoding processing unit 20f checks whether the decoded result is correct according to a predetermined check method. If an error is found in the data, an error correction function is used to calculate the correct data. The error correction function differs depending on the type of code.

[0080] In this embodiment, the decoding processing unit 20f decodes the code contained in the ideal image after the inference processing described below, but can also decode the code contained in the read image before the inference processing. The decoding processing unit 20f is configured to write the decoded result obtained by decoding the code to the decoded result storage unit 40b of the ROM 40 shown in Figure 3.

[0081] (Configuration of extraction unit) The extraction unit 20g extracts code candidate areas from the scanned image generated by the camera 5 that are likely to contain a code. The code candidate areas can be extracted based on features that indicate the likelihood of a code. In this case, the features that indicate the likelihood of a code serve as information for identifying the code. For example, the extraction unit 20g can acquire a scanned image and search for a code in the acquired scanned image based on the features that indicate the likelihood of a code. Specifically, the extraction unit 20g searches the acquired scanned image for a portion that has a predetermined or higher level of features that indicate the likelihood of a code. If a portion with a feature that indicates the likelihood of a code is found, the extraction unit 20g extracts the portion as a code candidate area. The code candidate area may include areas other than codes, but it must at least include a portion that is more likely to be a code than a predetermined level. Note that the code candidate area is merely an area where a code is likely to exist, and as a result, it may not contain a code.

[0082] The extraction unit 20g can also extract a code candidate region by using position information of the aimer light emitted from the aimer light emitting unit 10 as information for identifying a code. In this case, the extraction unit 20g identifies a portion of the read image corresponding to a mark formed by the aimer light emitting unit 10 and extracts the identified portion as a code candidate region. That is, as described above, the aimer light is used to indicate the shooting range and field of view center of the camera 5, so the mark formed by the aimer light can be aligned in advance with the field of view center of the camera 5. In particular, in the case of the handheld optical information reading device 1A, since the user aims the aimer light at the code to be read, there is a high probability that the mark formed by the aimer light will overlap with the code. That is, by aligning the portion corresponding to the mark formed by the aimer light emitting unit 10 with the field of view center of the camera 5, when the extraction unit 20g extracts a region including the field of view center of the camera 5, that region is likely to contain a code. In this case, the part corresponding to the mark formed by the aimer light emitting unit 10 does not have to be perfectly aligned with the center of the field of view of the camera 5, and may be slightly offset vertically or horizontally of the field of view. Furthermore, since the code has a predetermined size, the area extracted by the extraction unit 20g is not only the center of the field of view of the camera 5, but also an area of ​​a predetermined size that includes the center of the field of view.

[0083] The extraction unit 20g can also identify the center of the scanned image and extract the center as a code candidate area. For example, by acquiring the center of the field of view of the camera 5 in advance as information for identifying the code, the center of the field of view of the camera 5 can be identified on the scanned image. In particular, in the case of the handheld optical information reader 1A, the center identified on the scanned image corresponds to the mark formed by the aimer light emitting unit 10, and therefore the center is an area where a code is likely to exist.

[0084] The extraction unit 20g can be configured to accept a user's designation of a specific portion of the scanned image and extract the designated portion as a code candidate area. That is, when the user designates a region of any size at any position (coordinate) on the scanned image, the extraction unit 20g accepts the designation of the specific portion of the scanned image based on the coordinates and size information of the position. The extraction unit 20g extracts the accepted portion as a code candidate area. For example, in the case of a stationary optical information reading device 1, as shown in FIG. 1, a workpiece W being transported by a conveyor belt B is photographed and a code decoding process is performed. However, the workpiece W is not necessarily located in the center of the width direction of the conveyor belt B, but may be located at the edge. In this case, by designating a portion of the scanned image corresponding to the edge of the conveyor belt B, it is possible to accurately extract the region as a region where a code is likely to exist. In addition, there are cases where a code is displayed near the edge of a large workpiece W, away from the center. In such cases, by specifying a part of the scanned image that corresponds to the edge of the workpiece W, it is possible to accurately extract the area where the code is likely to be present.

[0085] (Storage device, ROM configuration) 3 can be configured as a readable / writable storage device such as an SSD (Solid State Drive), etc. The storage device 30 can store various programs, setting information, image data, etc.

[0086] The ROM 40 includes an image data storage unit 40a, a decoded result storage unit 40b, a parameter set storage unit 40c, and a neural network storage unit 40d. The image data storage unit 40a stores the scanned image generated by the camera 5. The decoded result storage unit 40b stores the decoded result of the code executed by the decode processing unit 20f. The parameter set storage unit 40c stores the results of tuning executed by the tuning execution unit 20d, various conditions set, and various conditions set by the user. The neural network storage unit 40d stores the structure and parameters of a trained neural network, which will be described later.

[0087] At least a part of the storage units 40a, 40b, 40c, and 40d may be provided in a portion other than the ROM 40, and may be provided in the storage device 30, for example.

[0088] (Inference processing using neural networks) The optical information reading device 1, 1A has an ideal image generation function that generates an ideal image by performing inference processing using a neural network on a read image acquired by the camera 5. In this embodiment, an example will be described in which, instead of performing machine learning in the optical information reading device 1, 1A, the optical information reading device 1, 1A stores in advance the structure and parameters of a neural network that has already completed learning, and performs inference processing using a neural network configured with the stored structure and parameters, but this is not limiting, and the optical information reading device 1, 1A may perform machine learning to change the structure and parameters of the neural network.

[0089] As shown in Fig. 6, the neural network has an input layer to which input data (image data in this example) is input, an output layer that outputs output data, and an intermediate layer provided between the input layer and the output layer. For example, multiple intermediate layers can be provided, thereby making it possible to create a neural network with a multi-layer structure.

[0090] (Neural network training) First, the basic procedure for training a neural network will be described with reference to the flowchart shown in FIG. 7. Training of the neural network can be performed using a computer prepared for training purposes other than the optical information reading device 1, 1A, but it may also be performed using a general-purpose computer other than that for training. Training may also be performed in the optical information reading device 1, 1A. Conventionally known methods may be used as the training method for the neural network.

[0091] After starting, in step SA1, data of a previously prepared defective image is read. A defective image is an image that has parts that are inappropriate for reading a code, and an example of such an image is shown on the left side of FIG. 8. An inappropriate part is, for example, a dirty part or a colored part. Then, proceeding to step SA2, data of an ideal image that has been prepared in advance is read. An ideal image is an image that is appropriate for reading a code, and an example of such an image is shown on the right side of FIG. 8. A defective image and an ideal image are paired, and multiple such pairs are prepared. When learning is performed using the optical information reading device 1, 1A, a defective image and an ideal image are input to the optical information reading device 1, 1A.

[0092] In step SA2, a loss function is calculated to determine the difference between the defective image and the ideal image. In step SA3, the neural network parameters are updated to reflect the difference determined in step SA2.

[0093] In step SA4, it is determined whether the machine learning completion condition is met. The machine learning completion condition can be set based on the difference calculated in step SA2. For example, if the difference is equal to or less than a predetermined value, it can be determined that the machine learning completion condition is met. If step SA4 returns YES and the machine learning completion condition is met, the machine learning is terminated. On the other hand, if step SA4 returns NO and the machine learning completion condition is not met, the process returns to step SA1, where another defective image is read, and then the process proceeds to step SA2, where another ideal image paired with the other defective image is read.

[0094] In this way, a trained neural network can be generated in advance by performing machine learning on a plurality of defective images and a plurality of ideal images corresponding to the plurality of defective images. By storing the structure and parameters of the trained neural network in the optical information reading device 1, 1A, a trained neural network can be constructed within the optical information reading device 1, 1A. As in this example, by performing machine learning outside the optical information reading device 1, 1A and storing the resulting neural network structure and parameters in the optical information reading device 1, 1A, it becomes possible to perform inference processing using a neural network while realizing a small and lightweight optical information reading device 1, 1A. Note that, if a processor 20 with sufficiently high processing power is installed, there is no problem with performing training within the optical information reading device 1, 1A.

[0095] Figure 9 is a conceptual diagram of a convolutional neural network (CNN) configured as described above. "Convolution" extracts the features of the input image. This layer is composed of convolution operations similar to image processing filters. The filter weights are called kernels, and feature extraction is performed according to the kernels. Convolution layers generally have multiple different kernels, and the number of maps (D) increases according to the number of kernels. The kernel size can be, for example, 3x3 or 5x5.

[0096] "Pooling" performs a reduction process to combine the responses of each kernel. "Deconvolution" uses a deconvolution filter to reconstruct an image from the feature values. "Unpooling" performs an expansion process to sharpen the response values.

[0097] The structure and parameters of the trained neural network are stored in the neural network storage unit 40d of the ROM 40 shown in Fig. 3. The structure of the neural network refers to the number of intermediate layers (D) provided between the input layer and the output layer, the number of image processing filters, etc. The parameters of the neural network are the parameters set in step SA3 of the flowchart shown in Fig. 7.

[0098] The neural network storage unit 40d can also store the structures and parameters of multiple neural networks with different numbers of layers or image processing filters. In this case, the structures and parameters of multiple neural networks can be stored in association with code conditions in the neural network storage unit 40d. For example, the structure and parameters of a first neural network can be associated with a first code condition corresponding to the first neural network. Since the optimal number of layers or filters of the first neural network can be determined in advance by the first code condition, particularly the first PPC, the association is performed so as to maintain this relationship. Similarly, the structure and parameters of a second neural network different from the first neural network can be associated with a second code condition different from the first code condition.

[0099] If the neural network storage unit 40d stores the structures and parameters of multiple neural networks, the tuning unit 20d operates as follows when setting the optical information reading device 1, 1A. That is, when setting the code conditions contained in the scanned image generated by the camera 5, the tuning unit 20d reads the structure and parameters of the neural network associated with the set code conditions from the neural network storage unit 40d and identifies it as the neural network to be used during operation. For example, if the tuning unit 20d sets a first code condition as the code condition contained in the scanned image, the tuning unit 20d reads the structure and parameters of the first neural network associated with the first code condition from the neural network storage unit 40d. Then, the tuning unit 20d identifies the structure and parameters of the first neural network as the neural network to be used during operation of the optical information reading device 1, 1A. This allows inference processing to be performed on the scanned image using the neural network structure and parameters optimal for the code conditions, thereby improving reading accuracy.

[0100] When training the neural network, it is also possible to train the neural network on defective images and ideal images whose pixel resolution of the modules constituting the code is within a specific range. The pixel resolution of the module can be expressed, for example, by the number of pixels (PPC) of one module constituting the code, and the neural network may be trained on only defective images and ideal images whose PPC is within a specific range.

[0101] The specific range includes pixel resolutions that provide a high effect in inference processing for scanned images, but excludes pixel resolutions that provide little improvement in the effect in inference processing for scanned images, and can be set assuming inference processing for images that include codes made up of multiple modules. This allows for improved processing speed as a neural network structure specialized for inference processing for images that include codes.

[0102] The specific range of pixel counts can be, for example, 4 PPC or more and 6 PPC or less, but it may also be less than 4 PPC or more than 6 PPC. Also, by limiting the specific range to 4 PPC or more and 6 PPC or less for machine learning, it is possible to eliminate unnecessary scale variations from the images used for learning and optimize the trained neural network structure.

[0103] (inference processing) As shown in Fig. 5, the processor 20 of the optical information reader 1, 1A is provided with an inference processing unit 20j configured with a dedicated core 25. The inference processing unit 20j reads out the structure and parameters of a neural network stored in the neural network storage unit 40d of the ROM 40, and configures a neural network in the optical information reader 1, 1A using the read-out structure and parameters of the neural network. The neural network configured in the optical information reader 1, 1A is the same as the trained neural network trained according to the procedure shown in the flowchart of Fig. 7.

[0104] The inference processing unit 20j executes an inference process to input the scanned image generated by the camera 5 into a neural network, thereby generating an ideal image corresponding to the scanned image in accordance with the structure and parameters of the neural network stored in the neural network storage unit 40d. Since the ideal image generated by the inference processing unit 20j is merely an image generated by the inference process, it is not necessarily the same as the ideal image used during the learning described above, but in this embodiment, the image generated by the inference processing unit 20j is also referred to as an ideal image.

[0105] The filter processing unit 20c is configured to apply an image processing filter to the scanned image generated by the camera 5 before the inference processing unit 20j inputs the scanned image to the neural network. This allows appropriate image processing to be performed before inference processing by the neural network is performed, resulting in more accurate inference processing.

[0106] The inference processing unit 20j may perform inference processing on all read images to be decoded by the decoding processing unit 20f, or the inference processing unit 20j may perform inference processing on only some of the read images to be decoded by the decoding processing unit 20f. For example, the decoding processing unit 20f performs decoding processing on the read image before performing inference processing, and determines whether the decoding was successful. If the decoding is successful, this means that the read image did not require inference processing, and the result is output as is. On the other hand, if the decoding fails, the inference processing unit 20j performs inference processing, and the decoding processing unit 20f performs decoding processing on the generated ideal image.

[0107] (Input partial image) The image input by inference processing unit 20j to the neural network may be the entire scanned image generated by camera 5, but the larger the size of the input image, the greater the amount of calculations by the neural network, which increases the calculation load on processor 20 and may ultimately result in a decrease in processing speed. Specifically, in an FCN (Fully Convolutional Network) type neural network as shown in Figure 9, assuming the structure is the same, the amount of calculations is proportional to the input image size, and since the input image size is proportional to the square of the code size, the amount of calculations is also proportional to the square of the code size.

[0108] On the other hand, there are virtually no cases where a code is present throughout the entire scanned image generated by camera 5. During normal operation, a code is present only in a small area of ​​the scanned image, and if it is possible to perform inference processing using a neural network only in that area, reading accuracy can be improved.

[0109] Therefore, the image input by the inference processing unit 20j to the neural network can be a portion of the scanned image generated by the camera 5, i.e., a partial image containing a code. Specifically, the partial image corresponding to the code candidate region extracted by the extraction unit 20g is stored in the image data storage unit 40a and made accessible to the inference processing unit 20j. The inference processing unit 20j inputs the partial image to the neural network, and performs inference processing on the partial image according to the structure and parameters stored in the neural network storage unit 40d. The decoding processing unit 20f then performs decoding processing on the generated ideal image. Furthermore, as will be described later, when performing decoding processing without performing inference processing on the partial image, the decoding processing unit 20f makes the image data storage unit 40a accessible, and the decoding processing unit 20f reads the partial image stored in the image data storage unit 40a and performs decoding processing.

[0110] As mentioned above, it is rare for a code to be present in the entire scanned image, so the size of the partial image corresponding to the code candidate area is smaller than the size of the scanned image. This reduces the size of the image input to the neural network, reducing the computational load on the processor 20 and ultimately achieving faster processing speeds. Furthermore, because the partial image corresponds to an area where a code is likely to be present, it is possible to create an image that includes the information necessary for reading, i.e., the entire code. Because this image including the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

[0111] The inference processing unit 20j can determine the input size of the partial image to the neural network based on the code size acquired by the tuning execution unit 20d. As described above, the tuning execution unit 20d can acquire the code size. The inference processing unit 20j increases the input size of the partial image to be input to the neural network by a predetermined amount compared to the code size acquired by the tuning execution unit 20d.

[0112] That is, while inputting a partial image into the neural network can reduce the computational load, if the code is rotated or the position of the code cannot be accurately detected, a portion of the code in the partial image may be missing, potentially resulting in a failure in the decoding process. In this embodiment, the input size to the neural network is not the same as the code size acquired by the tuning execution unit 20d, but is larger than the code size by a predetermined amount, thereby preventing a portion of the code from being missing from the partial image. By making the input size larger than the code size by a predetermined amount, an upper limit can be placed on the input size to the neural network, thereby preventing the computational load from becoming too heavy. The "predetermined amount" may mean, for example, that the horizontal (or vertical) length of the partial image input to the neural network is 1.5 times or more, 2.0 times or more, or 3.0 times or more the horizontal (or vertical) length of the code acquired by the tuning execution unit 20d.

[0113] When the inference processing unit 20j determines the input size of the partial image to be input to the neural network, it can determine the size based on the number of pixels in one module that makes up the code acquired by the tuning execution unit 20d and the number of modules arranged vertically or horizontally in the code.

[0114] Furthermore, the tuning execution unit 20d can set the code conditions as described above. The inference processing unit 20j can also set the size of the scanned image to be input to the neural network based on the code conditions set by the tuning execution unit 20d. For example, when the inference processing unit 20j acquires the code size from among the code conditions set by the tuning execution unit 20d, it inputs a partial image that is a predetermined amount larger than the acquired code size to the neural network. If the code size is large, the size of the partial image to be input to the neural network will be large, and if the code size is small, the size of the partial image to be input to the neural network will be small. In other words, the size of the partial image to be input to the neural network can be changed according to the code conditions.

[0115] (Zoom in / out function) In order for a convolutional neural network like the one shown in Figure 9 to fully capture the features of the code and create an ideal image suitable for decoding, it is necessary to extract features within a range covering a certain number of modules. For example, if a feature value obtained from the neural network is a value extracted from a 6x6 module range, and an image is captured with one module consisting of 20 pixels, the neural network must be designed to calculate a feature value aggregated from a 120x120 pixel range. In other words, the wider the pixel range to be covered, the deeper the neural network hierarchy must be. For example, to aggregate feature values ​​from the above-mentioned 120x120 pixel range, six convolutional layers are required.

[0116] However, because the code is composed of a random arrangement of white and black modules, the relationship between pixel values ​​over a wide range is weak, and expanding the pixel range beyond a certain point hardly improves the inference processing effectiveness. Such features are referred to as narrow-range features in this specification.

[0117] Furthermore, the arrangement of fixed module patterns, such as finder patterns, contained in code, or the rectangular shape of the code as a whole can be broad-range features, but if you only roughly separate the modules from the background, inference processing can be performed sufficiently using only narrow-range features, without using broad-range features. A broad-range feature can be defined as a feature that has a fixed shape over a wide range.

[0118] Therefore, when training the neural network shown in Figure 7, images in which the PPC of the code is within a specific range can be used. While this can improve the processing speed as a neural network structure specialized for inference processing of images containing codes, there is a concern that the effectiveness of inference processing for scanned images in which the PPC is outside the specific range may be reduced. Note that when training the neural network, images in which the PPC of the code is outside the specific range may also be used.

[0119] The optical information reading devices 1 and 1A of this embodiment are provided with a function for reducing and enlarging a read image so that the PPC of the read image falls within a specific range. As shown in Fig. 5, the processor 20 is configured with a reduction unit 20h and an enlargement unit 20i. The reduction unit 20h is a part that generates a reduced read image so that the pixel resolution of the modules that make up the code in the read image generated by the camera 5 falls within the specific range, and the enlargement unit 20i is a part that generates an enlarged read image so that the pixel resolution of the modules that make up the code in the read image generated by the camera 5 falls within the specific range.

[0120] The reduction unit 20h and the enlargement unit 20i determine whether the PPC of the code in the scanned image generated by the camera 5 is outside a specific range (4 PPC to 6 PPC), and if it is within the specific range, they do not perform reduction or enlargement, but if it is outside the specific range, they perform reduction or enlargement. By performing reduction or enlargement, the PPC of the code in the partial image input to the neural network by the inference processing unit 20j falls within the specific range.

[0121] The inference processing unit 20j inputs the scanned image, enlarged or reduced to fit within a specific range, into a neural network configured with the structure and parameters stored in the neural network storage unit 40d, and performs inference processing on the scanned image in accordance with the structure and parameters.The decoding processing unit 20f then performs decoding processing on the generated ideal image.

[0122] The extraction unit 20g may be configured to search for a code in the scanned image enlarged by the enlargement unit 20i or reduced by the reduction unit 20h, and extract an area containing the searched code as a code candidate area where a code is likely to exist. In this case, the inference processing unit 20j inputs a partial image corresponding to the code candidate area extracted by the extraction unit 20g to a neural network configured with a structure and parameters stored in the neural network storage unit 40d, and performs inference processing on the scanned image in accordance with the structure and parameters.

[0123] When setting up the optical information reading device 1, 1A, the tuning execution unit 20d can generate multiple read images (enlarged images) with different magnification ratios and multiple read images (reduced images) with different reduction ratios. The tuning execution unit 20d inputs the generated multiple read images (enlarged images or reduced images) into a neural network to perform inference processing for each read image, and performs decoding processing on each generated ideal image to determine a reading margin indicating the ease of code reading. The tuning execution unit 20d identifies the magnification or reduction ratio of the read image for which the determined reading margin is higher than a predetermined value as the magnification or reduction ratio to be used during operation. When the tuning execution unit 20d identifies the magnification or reduction ratio, during operation of the optical information reading device 1, 1A, the reduction unit 20h and the enlargement unit 20i enlarge or reduce the read image at the magnification or reduction ratio identified by the tuning execution unit 20d.

[0124] That is, for example, if the specific range has a certain width, it is conceivable that the reading margin will change if the magnification or reduction ratio is changed even within that specific range. Since the magnification or reduction ratio of the read image during operation can be specified so that the reading margin calculated by the tuning execution unit 20d is higher than a predetermined value, processing speed and reading accuracy are improved.

[0125] After determining the reading margin of each scanned image, the tuning execution unit 20d can specify the magnification or reduction ratio of the scanned image with the highest reading margin among the determined reading margins as the magnification or reduction ratio to be used during operation, thereby further improving processing speed and scanning accuracy.

[0126] Furthermore, by utilizing the ability of the reduction unit 20h and the enlargement unit 20i to reduce and enlarge the scanned image, the number of parameter sets can be increased. Multiple parameter sets with different reduction ratios by the reduction unit 20h and multiple parameter sets with different enlargement ratios by the enlargement unit 20i can be generated, and these parameter sets can be stored in the parameter set storage unit 40c of the ROM 40. For example, if multiple reduction ratios by the reduction unit 20h can be set, such as 1 / 2, 1 / 4, and 1 / 8, the parameter set with a reduction ratio of 1 / 2, a parameter set with a reduction ratio of 1 / 4, and a parameter set with a reduction ratio of 1 / 8 can be stored in the parameter set storage unit 40c. Furthermore, if multiple enlargement ratios by the enlargement unit 20i can be set, such as 2x, 4x, and 8x, the parameter set with a magnification ratio of 2x, a parameter set with a magnification ratio of 4x, and a parameter set with a magnification ratio of 8x can be stored in the parameter set storage unit 40c. Any one of the multiple parameter sets stored in the parameter set storage unit 40c can then be applied.

[0127] (inference processing filter) The inference processing unit 20j may be a part that executes an inference processing filter that executes inference processing using a neural network. The inference processing filter is a filter that executes inference processing according to the structure and parameters stored in the neural network storage unit 40d by inputting the scanned image to a neural network configured by the structure and parameters stored in the neural network storage unit 40d, and performs the same function as the inference processing function described above.

[0128] When the inference processing filter is made executable, a setting unit 20e (shown in FIG. 5) for setting the inference processing filter can be provided. The setting unit 20e is configured to be able to accept the setting of the inference processing filter by the user. For example, when setting up the optical information reading device 1, 1A, the setting unit 20e can be configured to generate a user interface that allows the user to select either "apply the inference processing filter" or "not apply the inference processing filter" and display it on the display unit 6, and accept the user's selection. Therefore, the inference processing unit 20j applies the inference processing filter set by the setting unit 20e to the read image, thereby performing inference processing on the read image.

[0129] The setting unit 20e may be included in the tuning execution unit 20d, or may be configured separately from the tuning execution unit 20d. If the setting unit 20e is included in the tuning execution unit 20d, it becomes possible to set parameters related to the inference processing filter during tuning. The parameters related to the inference processing filter can include "apply inference processing filter" and "not apply inference processing filter." "Apply inference processing filter" means setting the inference processing filter to be executable, and "not apply inference processing filter" means not setting the inference processing filter.

[0130] The parameters related to the inference processing filter are also information related to the inference processing filter. In this case, a parameter set including information related to the setting of the inference processing filter can be stored in the parameter set storage unit 40c. The parameters related to the inference processing filter include "apply inference processing filter" and "not apply inference processing filter." Therefore, the parameter sets stored in the parameter set storage unit 40c include a first parameter set that sets the inference processing filter to be executable and a second parameter set that does not set the inference processing filter. When the optical information reading device 1, 1A is operated, one of the first parameter set and the second parameter set is applied. When the first parameter set is applied, the decoding processing unit 20f performs decoding on an ideal image on which inference processing has been performed by the inference processing filter. Meanwhile, when the second parameter set is applied, the decoding processing unit 20f performs decoding on a read image on which inference processing has not been performed.

[0131] The parameter sets stored in the parameter set storage unit 40c also include items for setting code conditions included in the scanned image generated by the camera 5. Items for setting code conditions include the PPC, code type, code size, etc. acquired by the tuning execution unit 20d. For example, one parameter set may include the PPC, code type, and code size as items for setting code conditions, the gain of the camera 5, the light intensity and exposure time of the illumination unit 4 as imaging conditions, and the type of image processing filter, parameters of the image processing filter, and application items of the inference processing filter as items of the image processing filter applied by the filter processing unit 20c. Values ​​set by the tuning execution unit 20d may be used as they are, or the user may change them as desired.

[0132] (An example of tuning process steps) An example of the procedure of the tuning process performed by the tuning execution unit 20d when setting up the optical information reading device 1, 1A will be specifically described with reference to the flowchart shown in Fig. 10. In step SB1 after the start of the flowchart shown in Fig. 10, the tuning execution unit 20d controls the illumination unit 4 and camera 5 to cause the camera 5 to generate a read image, and the tuning execution unit 20d acquires the read image. At this time, the presence and type of an image processing filter to be executed before the decoding process and the decoding process parameters for applying inference processing by a neural network are set to arbitrary parameters. Next, proceeding to step SB2, the tuning execution unit 20d causes the decoding processing unit 20f to execute decoding processing on the acquired read image.

[0133] After the decoding process, the process proceeds to step SB3, where the tuning execution unit 20d determines whether the decoding process of step SB2 was successful. If the determination in step SB3 is NO and the decoding process of step SB2 has failed, that is, if the code cannot be read, the process proceeds to step SB4, where the decoding process parameters are changed to other parameters, and the decoding process is executed again in step SB2. If the decoding process has failed with all of the decoding process parameters, this flow ends and the user is notified.

[0134] On the other hand, if the determination in step SB3 is YES and the decoding process in step SB2 is successful, the process proceeds to step SB5, where the tuning execution unit 20d determines code parameters (PPC, code type, code size, etc.). After the tuning execution unit 20d determines the code parameters, the structure and parameters of the neural network stored in the neural network storage unit 40d in association with the code parameters are also determined (step SB6). In step SB6, a neural network is configured using the structure and parameters read from the neural network storage unit 40d.

[0135] In step SB7, the inference processing unit 20j executes inference processing by inputting the read image into the neural network constructed in step SB6, and the decoding processing unit 20f executes decoding processing for the generated ideal image. Then, the process proceeds to step SB8, where the tuning execution unit 20d evaluates the reading margin based on the decoding processing result of step SB7 and temporarily stores the evaluation result.

[0136] In step SB9, it is determined whether the decoding process has been completed for all the decoding process parameters. If the determination in step SB9 is NO, meaning that the decoding process has not been completed for all the decoding process parameters, the process proceeds to step SB10, where the decoding process parameters are changed to other parameters, and then the process proceeds to step SB7.

[0137] On the other hand, if step SB9 returns YES and the decoding process is completed for all the decoding process parameters, the process proceeds to step SB11. In step SB11, the tuning execution unit 20d selects the decoding process parameter with the highest read margin from among all the decoding process parameters, and determines the selected decoding process parameter as the parameter to be applied during operation. Note that during the tuning process, the imaging conditions and the like are also set to appropriate conditions.

[0138] (Decoding procedure before determining reduction and enlargement ratios) Next, an example of a decoding process procedure before determining the reduction ratio and enlargement ratio will be specifically described with reference to the flowchart shown in Fig. 11. The process specified in the flowchart shown in Fig. 11 can be executed in step SB2 of the flowchart shown in Fig. 10.

[0139] 11, in step SC1 after the start, the tuning execution unit 20d selects an arbitrary image processing filter from among a plurality of image processing filters. This image processing filter is a filter to be executed by the filter processing unit 20c. Then, in step SC2, the filter processing unit 20c executes the image processing filter selected in step SC1 on the scanned image.

[0140] Next, the process proceeds to step SC3, where an image pyramid is created. The image pyramid is composed of the original read image, an image obtained by reducing the original read image by half, an image obtained by reducing the original read image by a quarter, an image obtained by reducing the original read image by an eighth, and so on. The reduction of the read image is performed by a reduction unit 20h. The image pyramid can also be composed of the original read image, an image obtained by enlarging the original read image by two times, an image obtained by enlarging the original read image by four times, an image obtained by enlarging the original read image by eight times, and so on. The enlargement of the read image is performed by an enlargement unit 20i. Note that either the reduced image or the enlarged image may be omitted.

[0141] After creating the image pyramid, the process proceeds to step SC4, where an arbitrary scanned image is selected from the multiple scanned images that make up the image pyramid. The selected images may include the original scanned image that has not been reduced or enlarged. In step SC5, the extraction unit 20g extracts a code candidate area that is likely to contain a code from the scanned image selected in step SC4. Then, the process proceeds to step SC6, where the inference processing unit 20j inputs the partial image corresponding to the code candidate area extracted in step SC5 into a neural network and executes inference processing. The size of the partial image to be input to the neural network is determined by the code conditions described above.

[0142] After performing the inference process to generate an ideal image, the process proceeds to step SC7, where a positioning process is performed on the outline of the code and the modules that make up the code. After the positioning process, the process proceeds to step SC8, where it is determined whether each module that makes up the code is white or black. After determining whether it is black or white, the process proceeds to step SC9, where the decoding processing unit 20f performs a decoding process on the generated ideal image. In the decoding process, for example, a method of restoring a character string from a 0-1 matrix of the modules can be used.

[0143] Thereafter, the process proceeds to step SC10, where the tuning execution unit 20d determines whether the decoding process of step SC9 was successful. If the determination in step SC10 is NO and the decoding process of step SC9 has failed, the process proceeds to step SC11, where it is determined whether all the read images constituting the image pyramid have been selected. If the determination in step SC11 is NO and all the read images constituting the image pyramid have not been selected, the process proceeds to step SC4, where another reduced read image or another enlarged read image constituting the image pyramid is selected, and the process proceeds to step SC5.

[0144] On the other hand, if the determination in step SC11 is YES and all the scanned images constituting the image pyramid have been selected, this flow ends and the conditions under which the decoding was successful are stored.

[0145] (Decoding procedure after determining reduction and enlargement ratios) Next, an example of a decoding process procedure after determining the reduction ratio and enlargement ratio will be specifically described with reference to the flowchart shown in Fig. 12. The process specified in the flowchart shown in Fig. 12 can be executed in step SB7 of the flowchart shown in Fig. 10 and when the optical information readers 1 and 1A are in operation.

[0146] In step SD1 after the start of the flowchart shown in Fig. 12, the filter processing unit 20c applies the image processing filter selected in step SC1 of the flowchart shown in Fig. 11 to the read image. At this time, if the filter processing unit 20c applies an averaging filter to the read image shown in the upper part of Fig. 13, an image after application of the image processing filter as shown in the lower part of Fig. 13 is obtained.

[0147] Then, the process proceeds to step SD2, where the scanned image is reduced or enlarged as necessary. Specifically, if the PPC of the code in the scanned image after the image processing filter is applied is within a specific range, reduction or enlargement is not performed, but if it is outside the specific range, reduction or enlargement is performed so that the PPC falls within the specific range. An example of the scanned image after reduction processing is shown in the upper part of Figure 14.

[0148] Next, in step SD3, the extraction unit 20g extracts a code candidate area that is likely to contain a code from the scanned image reduced or enlarged in step SD2. An example of an extracted code candidate area image is shown in the lower part of Fig. 14. Note that if the scanned image is not reduced or enlarged in step SD2, the extraction process is performed on the original scanned image in step SD3.

[0149] Then, the process proceeds to step SD4, where the inference processing unit 20j inputs the partial image corresponding to the code candidate region extracted in step SD3 into the neural network and executes the inference process. Figure 15 shows an example of the scanned image before the inference process and the ideal image after the inference process. After the ideal image is generated, the process goes through steps SD5 to SD7 and ends this flow. Steps SD5 to SD7 are the same as steps SC7 to SC9 in the flowchart shown in Figure 11.

[0150] (Two types of decoding) As described above, in this embodiment, there are two cases: one in which the decoding processing unit 20f decodes the read image generated by the camera 5 without the inference processing unit 20j performing inference processing on the read image, and one in which the inference processing unit 20j performs inference processing on the read image generated by the camera 5 to generate an ideal image, and the decoding processing unit 20f decodes the generated ideal image. The former case, i.e., the case in which the read image is decoded, is referred to as the first decoding process, and the latter case, i.e., the case in which the ideal image generated by the inference processing is decoded, is referred to as the second decoding process.

[0151] The first decoding process and the second decoding process are both executed by the decoding processing unit 20f, but may be executed by different decoding processing units. That is, since the processor 20 has the dedicated core 25 and the first to fourth general-purpose cores 21 to 24, the inference process by the inference processing unit 20j configured by the dedicated core 25 and the first decoding process by the decoding processing unit 20f configured by at least one of the first to fourth general-purpose cores 21 to 24 can be executed in parallel and at high speed. Furthermore, the processor 20 can execute the second decoding process by the decoding processing unit 20f after the inference process by the inference processing unit 20j is completed. This will be explained in detail below.

[0152] (First example) 16 is a flowchart showing a first example of a case where the first decoding process and the inference process are executed in parallel. In step SE1 after the start of this flowchart, the processor 20 acquires a read image. Then, in step SE2, the first decoding process is executed, that is, the decoding processing unit 20f executes a decoding process on the read image. After the first decoding process, the process proceeds to step SE3, where it is determined whether or not the reading was successful. If the reading was not successful, the process returns to step SE2, and the first decoding process is executed again. If the reading is not successful after a predetermined number of times (time) has elapsed, a timeout occurs, and the decoding process is terminated. On the other hand, if it is determined in step SE3 that the reading was successful, the decoding process is terminated.

[0153] In addition, while proceeding from step SE1 to step SE2, the process proceeds to step SE4 in parallel. In step SE4, the inference processing unit 20j performs inference processing on the read image generated by the camera 5 to generate an ideal image. The generation of this ideal image and the first decoding processing of step SE2 are performed in parallel and are separate read images. After the ideal image is generated in step SE4, the process proceeds to step SE5, where the decoding processing unit 20f performs second decoding processing, i.e., decoding processing of the ideal image generated in step SE4. After the second decoding processing, the process proceeds to step SE6, where it is determined whether reading was successful. If reading was not successful, the process returns to step SE5 and performs the second decoding processing again. If reading is not successful after a predetermined number of times (time) has elapsed, a timeout occurs and the decoding processing ends. On the other hand, if it is determined in step SE6 that reading was successful, the decoding processing ends.

[0154] The optical information reader 1 may be provided with a function to cancel steps SE4 to SE6. Steps SE4 to SE6 are steps for executing the inference process and the second decoding process. Since the inference process takes time, canceling steps SE4 to SE6 can speed up the process. Steps SE4 to SE6 can be canceled, for example, by a user operation. For example, a selection parameter for whether or not to execute the inference process may be included as one of the parameters stored in the parameter set storage unit 40c. The user can change this selection parameter to switch between a mode in which the inference process is executed and a mode in which it is not executed. The selection parameter may be, for example, a selection parameter for a "speed priority mode." When the "speed priority mode" is selected by the user, the device operates in a mode in which the inference process is not executed. On the other hand, when the "speed priority mode" is not selected, the device operates in a mode in which the inference process is executed. Note that the inference process may be automatically canceled when it is determined that the inference process is unnecessary based on information obtained by tuning.

[0155] 17A is a sequence diagram showing sequence example 1 of the first example. As shown in this diagram, while at least one of the first to fourth general-purpose cores 21 to 24 is executing the first decoding process, the dedicated core 25 executes the inference process. After the first decoding process and the inference process are completed, the general-purpose core executes the second decoding process.

[0156] 17B is a sequence diagram showing sequence example 2 of the first example. In this example, the first general-purpose core 21 executes the first decoding process and the second decoding process in parallel in separate threads. Specifically, the dedicated core 25 executes the inference process while the first general-purpose core 21 executes the first decoding process. When the inference process ends, the first general-purpose core 21 executes the second decoding process in a separate thread from the first decoding process without waiting for the first decoding process to end.

[0157] 17C is a sequence diagram showing sequence example 3 of the first example. In this example, the first general-purpose core 21 executes the first decoding process, and the second general-purpose core 22 executes the second decoding process. Specifically, while the first general-purpose core 21 executes the first decoding process, the dedicated core 25 executes the inference process. When the inference process ends, the second general-purpose core 22 executes the second decoding process without waiting for the first decoding process to finish. This allows the first decoding process and the second decoding process to be executed by different general-purpose cores, thereby further speeding up the processing.

[0158] (Second example) 18 is a flowchart showing a second example of the case where the first decoding process and the inference process are executed in parallel. This second example shows the case where multiple code candidate areas are extracted. In step SF1 after the start of this flowchart, the extraction unit 20g executes an extraction process to extract code candidate areas that are likely to contain a code from the scanned image generated by the camera 5. This extracts a candidate area group consisting of multiple code candidate areas. The multiple code candidate areas are extracted from a single scanned image, and there may be areas that do not actually contain a code mixed in.

[0159] After step SF1, the process is divided into processing by the first decoding sequence processing unit and processing by the second decoding sequence processing unit. In step SF2, the first decoding sequence processing unit selects one code candidate area from the group of code candidate areas. At this time, the areas are ranked from most likely to contain a code to least likely, and the area most likely to contain a code is selected first. In step SF3, the decoding processing unit 20f performs the first decoding process on the code candidate area selected in step SF2. After the first decoding process, the process proceeds to step SF4, where it is determined whether reading was successful. If it is determined in step SF4 that reading was successful, the decoding process ends.

[0160] On the other hand, if reading is not successful, the process returns to step SF2, where one code candidate area that has not yet undergone the first decoding process is selected from the group of code candidate areas in the above order. Then, the process proceeds to step SF3, where the decoding processing unit 20f performs the first decoding process on the code candidate area selected in step SF2. If reading is again not successful, the process returns to step SF2. If reading is not successful, the decoding processing unit 20f repeats this process and performs the first decoding process on all code candidate areas. Even if reading is not successful at all, a timeout occurs and the decoding process ends.

[0161] Meanwhile, in step SF5, the second decoding sequence processing unit selects one code candidate area from the group of code candidate areas, similar to step SF2. In step SF6, the inference processing unit 20j performs inference processing on the code candidate area selected in step SF5 to generate an ideal image. This ideal image generation and the first decoding processing of step SF3 are performed in parallel. After the ideal image is generated in step SF6, the process proceeds to step SF7, where the decoding processing unit 20f performs second decoding processing, i.e., decoding processing of the ideal image generated in step SF6. After the second decoding processing, the process proceeds to step SF8, where it is determined whether reading was successful. If it is determined in step SF8 that reading was successful, the decoding processing ends.

[0162] On the other hand, if reading is not successful, the process returns to step SF5, and selects one code candidate area from the group of code candidate areas that has not undergone inference processing, in the order described above. Then, the process proceeds to step SF6, where an ideal image is generated, and then to step SF7. In step SF7, a second decoding process is performed. If reading is again not successful, the process returns to step SF5. If reading is not successful, this process is repeated to generate an ideal image and perform the second decoding process on all code candidate areas. The decoding process also ends if none of the code candidate areas are read successfully.

[0163] 19 is a sequence diagram showing a sequence example of the second example. In this example, both the first general-purpose core 21 and the second general-purpose core 22 execute the extraction process. As a result, even if there are multiple code candidate areas, the extraction process can be completed quickly. In this example, a case where code candidate areas 1 to 8 are extracted will be described.

[0164] After the extraction process, the first general-purpose core 21 executes a first decoding process on the code candidate area 1 (first decoding process 1). Furthermore, the second general-purpose core 22 executes a first decoding process on the code candidate area 2 (first decoding process 2). The first decoding process 1 and the first decoding process 2 are executed in parallel. Furthermore, the dedicated core 25 executes an inference process on the code candidate area 5 (inference process 1) while the first decoding process 1 and the first decoding process 2 are being executed.

[0165] When the first decoding process 1 is completed, the first general-purpose core 21 executes the first decoding process on the code candidate area 3 (first decoding process 3). When the first decoding process 2 is completed, the second general-purpose core 22 executes the first decoding process on the code candidate area 4 (first decoding process 4). When the dedicated core 25 finishes inference process 1 while executing the first decoding process 3 and the first decoding process 4, the dedicated core 25 executes inference process on the code candidate area 7 (inference process 2).

[0166] When the first decoding process 3 is completed, the first general-purpose core 21 executes the second decoding process (second decoding process 5) on the ideal image (image corresponding to code candidate area 5) generated by the inference process 1, since the inference process 1 has already been completed. On the other hand, when the first decoding process 4 is completed, the second general-purpose core 22 executes the first decoding process on the code candidate area 6, since the inference process 2 has not been completed.

[0167] The dedicated core 25 finishes inference process 2 while second decoding process 5 and first decoding process 6 are being executed. When first decoding process 5 finishes, inference process 2 has already finished, so the first general-purpose core 21 executes second decoding process on the ideal image (an image corresponding to code candidate region 7) generated by inference process 2 (second decoding process 7). On the other hand, when first decoding process 6 finishes, the second general-purpose core 22 executes first decoding process on code candidate region 8 (first decoding process 8). In this way, when an ideal image is generated by the inference process by the dedicated core 25, the first general-purpose core 21 and the second general-purpose core 22 are configured to execute the second decoding process in priority to the next first decoding process, without executing the next first decoding process, upon completion of the currently executed first decoding process.

[0168] Of the multiple general-purpose cores 21 to 24, for example, the first general-purpose core 21 may execute the extraction process, the second general-purpose core 22 may execute the first decoding process, and the third general-purpose core 23 may execute the second decoding process. Also, the fourth core 24 may execute image capture control for the camera 5 and control for the lighting unit 4. By dividing multiple types of processing among the general-purpose cores 21 to 24 in this way, processing can be further accelerated.

[0169] (Third example) 20 is a flowchart showing a third example of the case where the first decoding process and the inference process are executed in parallel. In this third example, when multiple code candidate areas are extracted, a step of determining whether or not the code candidate areas have been decoded is performed, and if it is determined that the code candidate areas have not been decoded, the first decoding process is performed.

[0170] In this flowchart, after starting, the process is divided into processing by the first decoding sequence processing unit and processing by the second decoding sequence processing unit. In step SG1 of the first decoding sequence processing unit, the extraction unit 20g executes a first extraction process to extract a code candidate area that is likely to contain a code from the read image generated by the camera 5. On the other hand, in step SG7 of the second decoding sequence processing unit, the extraction unit 20g executes a second extraction process to extract a code candidate area that is likely to contain a code from the read image generated by the camera 5.

[0171] The first extraction process in step SG1 and the second extraction process in step SG7 have different extraction conditions, and as will be described later, a first decoding process is executed for the code candidate region extracted in the first extraction process (step SG4), and an inference process is executed for the code candidate region extracted in the second extraction process (step SG14). The reason why the extraction conditions are different between the first extraction process and the second extraction process is that the aim of the first decoding sequence processing unit is to quickly capture and read images that are easy to read, while the aim of the second decoding sequence processing unit is to accurately capture and read images that are difficult to read and that are likely to fail in the first decoding sequence processing unit.

[0172] For example, the first extraction process extracts code candidate regions under a first predetermined condition that enables high-speed extraction, while the second extraction process extracts code candidate regions under a second predetermined condition that enables more accurate extraction than the first predetermined condition. In other words, the first predetermined condition enables faster extraction because it provides lower extraction accuracy than the second predetermined condition.

[0173] Specific condition settings include setting the extraction threshold of the first predetermined condition lower than the extraction threshold of the second predetermined condition, setting the number of extraction conditions constituting the first predetermined condition less than the number of extraction conditions constituting the second predetermined condition, etc. Also, the second predetermined condition may be configured by combining an extraction condition constituting the first predetermined condition with another extraction condition.

[0174] After performing high-speed extraction processing in step SG1, the process proceeds to step SG2, where one code candidate area is selected from the group of code candidate areas. Then, the process proceeds to step SG3, where it is determined whether the code candidate area selected in step SG2 is a code candidate area that has already been decoded. If the determination in step SG3 is YES and the code candidate area is already decoded, the process returns to step SG2, where another code candidate area is selected from the group of code candidate areas, and the process proceeds to step SG3.

[0175] If the determination in step SG3 is YES, this means that the code candidate area selected in step SG2 has not yet been decoded, so the process proceeds to step SG4, where the decoding processing unit 20f executes a first decoding process on the code candidate area selected in step SG2. After the first decoding process, the process proceeds to step SG5, where it is determined whether reading was successful. If it is determined in step SG5 that reading was successful, the process proceeds to step SG6, where it is determined whether reading has ended. If reading has ended in the first decoding process in step SG5, this flow ends, but if reading has not ended, the process returns to step SG2, selects another code candidate area from the group of code candidate areas, and proceeds to step SG3. If reading is not successful in step SG5, the process returns to step SG2.

[0176] On the other hand, when the second decoding sequence processing section proceeds to step SG8 via step SG7, one code candidate area is selected from the group of code candidate areas. Steps SG9 to SG12 are similar to steps SG3 to SG6 above.

[0177] Furthermore, when the process proceeds to step SG13 via step SG7, one code candidate area is selected from the group of code candidate areas. In step SG14, the inference processing unit 20j performs inference processing on the code candidate area selected in step SG13 to generate an ideal image. This ideal image generation and the first decoding processing of step SG4 and / or step SG10 are performed in parallel. After generating the ideal image in step SG14, the process proceeds to step SG15 to perform second decoding processing. After the second decoding processing, the process proceeds to step SG16 to determine whether reading was successful. If it is determined in step SG16 that reading was successful, it is determined in step SG17 whether reading has ended. If reading has ended in the first decoding processing of steps SG4 and SG10 or the second decoding processing of step SG15, this flow ends. However, if reading has not ended, the process returns to step SG13, selects another code candidate area from the group of code candidate areas, and proceeds to step SG14. If reading is not successful in step SG16, the process returns to step SG13.

[0178] 21 is a sequence diagram showing an example of a sequence of the third example. In this example, the first general-purpose core 21 executes a relatively high-speed first extraction process, and the second general-purpose core 22 executes a relatively high-precision second extraction process, so the time for the second extraction process is longer than the time for the first extraction process. In this example, a case will be described in which code candidate regions 1 to 9 are extracted by the first extraction process and the second extraction process.

[0179] After the first extraction process, the first general-purpose core 21 executes a first decoding process on code candidate area 1 (first decoding process 1). When the first decoding process 1 is completed, the first general-purpose core 21 executes a first decoding process on code candidate area 2 (first decoding process 2). After the second extraction process, the second general-purpose core 22 executes a first decoding process on code candidate area 3 (first decoding process 3). The first decoding process 2 and the first decoding process 3 are executed in parallel. After the second extraction process, the dedicated core 25 executes an inference process on code candidate area 6 (the code candidate area extracted in the second extraction process) while the first decoding process 2 and the first decoding process 3 are being executed (inference process 1).

[0180] Furthermore, when the first decoding process 2 is completed, the first general-purpose core 21 executes the first decoding process on the code candidate area 4 (first decoding process 4). When the first decoding process 3 is completed, the second general-purpose core 22 executes the first decoding process on the code candidate area 5 (first decoding process 5). When the dedicated core 25 finishes inference process 1 while executing the first decoding process 4 and the first decoding process 5, the dedicated core 25 executes inference process on the code candidate area 8 (inference process 2).

[0181] When the first decoding process 4 is completed, the first general-purpose core 21 executes the second decoding process (second decoding process 6) on the ideal image (image corresponding to code candidate area 6) generated by the inference process 1, since the inference process 1 has already been completed. On the other hand, when the first decoding process 5 is completed, the second general-purpose core 22 executes the first decoding process (first decoding process 7) on the code candidate area 7, since the inference process 2 has not yet been completed.

[0182] The dedicated core 25 finishes inference process 2 while the second decoding process 6 and the first decoding process 7 are being executed. When the second decoding process 6 finishes, the first general-purpose core 21 executes the second decoding process (second decoding process 8) on the ideal image (image corresponding to code candidate region 8) generated by inference process 2, since inference process 2 has already finished. On the other hand, when the first decoding process 7 finishes, the second general-purpose core 22 executes the first decoding process (first decoding process 9) on code candidate region 9.

[0183] (Task sequence description) 22 is a task sequence diagram when reading is successful in the first decoding process. This diagram shows a control task, a first decoding process task, a second decoding process task, and an inference process task. When the control task issues a read execution command 1 to the first decoding process task, the first decoding process begins. When reading is successful in the first decoding process, a read success notification 2 is issued to the control task. Upon receiving the read success notification 2, the control task issues a process end command 3 to the first decoding process task, and the first decoding process task issues a process end notification 4 to the control task.

[0184] Furthermore, when the control task issues a read execution instruction 5 to the second decoding task, the second decoding task issues an inference start instruction 6 to the inference task. When the inference task completes the inference process, it issues an inference completion notification 7 to the second decoding task. Thereafter, the second decoding task issues the next inference start instruction 8 to the inference task. After that, a processing end notification 4 is issued to the control task, so the control task issues a processing end instruction 9 to the second decoding task. The second decoding task issues an inference end instruction 10 to the inference task, and the inference task issues an inference end notification 11 to the second decoding task. Next, the second decoding task issues a processing end notification 12 to the control task to end reading. In other words, if decoding is successful in the first decoding process, the processor 20 is configured to end the inference process midway even if the inference process by the inference processing unit 20j has not yet been completed. This allows the next inference process to be started early.

[0185] Figure 23 is a task sequence diagram when reading is successful in the second decoding process. The control task issues a read execution command 1 to the first decoding process task, and a read execution command 2 to the second decoding process task. The second decoding process task issues an inference start command 3 to the inference process task. When the inference process is complete, the inference process task issues an inference completion notification 4 to the second decoding process task. After that, the second decoding process task issues the next inference start command 5 to the inference process task.

[0186] The second decoding task starts the second decoding process upon receiving the inference completion notification 4. When reading is successful in the second decoding process and the second decoding task issues a read success notification 6 to the control task, the control task issues a processing end command 7 to the second decoding task. At this time, the control task also issues a processing end command 8 to the first decoding task.

[0187] When the second decoding task receives a processing end command 7, it issues an inference end command 9 to the inference processing task, and the inference processing task issues an inference end notification 10 to the second decoding task. In addition, the first decoding task issues a processing end notification 11 to the control task. In addition, the second decoding task issues a processing end notification 12 to the control task. In this example, if decoding is successful in the second decoding process, the inference processing by the inference processing unit 20j can be terminated.

[0188] Figure 24 is a task sequence diagram when reading is successful in the first decoding process and the second decoding process. The control task issues a read execution command 1 to the first decoding process task, and a read execution command 2 to the second decoding process task. The second decoding process task issues an inference start command 3 to the inference process task. The first decoding process task starts the first decoding process. When reading is successful in the first decoding process and the first decoding process task issues a read success notification 4 to the control task, the control task issues a processing end command 5 to the second decoding process task. The second decoding process task issues a processing end notification 6 to the control task.

[0189] Meanwhile, when the inference processing task completes the inference processing, it issues an inference completion notification 7 to the second decoding processing task. The second decoding processing task then issues the next inference start command 8 to the inference processing task. The second decoding processing task starts the second decoding processing upon receiving the inference completion notification 7. When reading is successful in the second decoding processing and the second decoding processing task issues a read success notification 9 to the control task, the control task issues a processing end command 10 to the second decoding processing task.

[0190] When the second decoding task receives a processing end command 10, it issues an inference end command 11 to the inference processing task, and the inference processing task issues an inference end notification 12 to the second decoding task. After receiving the inference end notification 12, the second decoding task issues a processing end notification 13 to the control task. In this example, if decoding is successful in both the first decoding process and the second decoding process, the inference processing by the inference processing unit 20j can be terminated.

[0191] Figure 25 is a task sequence diagram when reading is successful twice in the second decoding process. The control task issues read execution command 1 to the first decoding process task, and read execution command 2 to the second decoding process task. The second decoding process task issues inference start command 3 to the inference process task. When the inference process is complete, the inference process task issues inference completion notification 4 to the second decoding process task. The second decoding process task then issues the next inference start command 5 to the inference process task. The second decoding process task starts the second decoding process upon receiving inference completion notification 4. Reading is successful in the second decoding process, and the second decoding process task issues a read success notification 6 to the control task.

[0192] Thereafter, the inference processing task completes the second inference processing and issues a second inference completion notification 7 to the second decoding processing task. The second decoding processing task receives the second inference completion notification 7 and starts the second second decoding processing. Reading is also successful in the second second decoding processing, and the second decoding processing task issues a read success notification 8 to the control task. During this time, reading is not successful in the first decoding processing task, so no read success notification is issued from the first decoding processing task.

[0193] The control task issues a processing end command 9 to the second decoding task. When the second decoding task receives the processing end command 9, it issues an inference end command 10 to the inference task, and the inference task issues an inference end notification 11 to the second decoding task. After receiving the inference end notification 11, the second decoding task issues a processing end notification 12 to the control task. Furthermore, when the control task issues a processing end command 13 to the first decoding task, the first decoding task issues a processing end notification 14 to the control task. In this example, if decoding is successful in both the first second decoding process and the second second decoding process, the inference processing by the inference processing unit 20j can be terminated.

[0194] (Continuous shooting with camera) As shown in Figure 1, when the optical information reading device 1 is used at a site where multiple works W are continuously conveyed, the codes will pass in front of the camera 5 in sequence at certain time intervals. For this reason, the camera 5 takes continuous photographs at predetermined short intervals to acquire the read images in sequence, and the optical information reading device 1 must complete reading within a set time.

[0195] Figure 26 is a reading sequence diagram in which image capture and decoding are repeated for a certain period of time after receiving a read start trigger signal. The time required for one decoding process is preset and is referred to as the decode timeout time. The total reading time limit is preset separately from the decode timeout time and is referred to as the tact timeout time. There are three timings when you might want to forcefully terminate the decoding process: when the decoding process is successful (timing 1), when the decode timeout time has elapsed since the start of reading (timing 2), and when the tact timeout time has elapsed since the start of reading (timing 3). For example, if the response speed to a forced termination command for the reading process is set to about 10 msec, it may be desirable to be able to terminate the decoding process within a similar amount of time.

[0196] On the other hand, since the inference process is executed by the dedicated core 25 and may require processing time of tens to hundreds of msec, it may be preferable to have either or both of the following functions: a function to terminate the inference process midway, or a function to start a new inference process midway.

[0197] Fig. 27 is a decoding process sequence diagram during continuous shooting. This decoding process sequence diagram is similar to the sequence diagram shown in Fig. 19, but shows an example in which the inference process is interrupted when one of the above timings 1 to 3 arrives during the inference process.

[0198] (Effects of the embodiment) As described above, according to this embodiment, when the optical information reading device 1, 1A is in operation, if the read image generated by the camera 5 is input to the neural network, an ideal image is generated by inference processing. Since a decoding process is performed on the ideal image after this inference processing, reading accuracy is improved.

[0199] Furthermore, if the scanned image is suitable for reading a code, the scanned image generated by the camera 5 is decoded without inference processing. Since the above inference processing and the decoding processing for decoding without inference processing are executed in parallel, it is possible to execute the decoding processing for another scanned image that does not require inference processing while executing the inference processing for a scanned image that requires inference processing, or to execute the inference processing for another scanned image that requires inference processing while executing the decoding processing for a scanned image that does not require inference processing. This reduces the processing time.

[0200] (Other embodiments) 28 is a diagram illustrating an example of a processing procedure during operation of an optical information reading device 1 according to another embodiment. In this embodiment, the processor 20 includes, in addition to the inference processing unit (image restoration unit) 20j, a pre-processing unit 200, a post-processing unit 201, a first decoding processing unit 202, and a second decoding processing unit 203. The pre-processing unit 200 is a unit that applies a noise reduction filter, a contrast correction filter, an averaging filter, etc. to the read image generated by the camera 5, similar to the filter processing unit 20c.

[0201] The first decoding processing unit 202 is a part that executes decoding processing on the scanned image generated by the camera 5, and the decoding processing by the first decoding processing unit 202 is also referred to as the first decoding processing. When the processor 20 has a plurality of cores (N cores), the first decoding processing unit 202 is configured by cores 0 to i. The first decoding processing unit 202 may also be configured by a single core.

[0202] The second decoding unit 203 is a unit that executes decoding processing on the restored image generated by the inference processing unit 20j, and the decoding processing by the second decoding unit 203 is also referred to as second decoding processing. If the processor 20 has N cores, the second decoding unit 203 is configured by core i+1 to core N-1. The second decoding unit 203 may also be configured by a single core.

[0203] 29, when the first decoding processing unit 202 is made up of cores 0 to i, each of cores 0 to i extracts a code candidate area where a code is likely to exist from the scanned image generated by camera 5, performs positioning processing of the code outline and the modules that make up the code, and then determines whether each module that makes up the code is white or black and performs decoding processing. The processing result is output to the post-processing unit 201.

[0204] On the other hand, when the second decoding processing unit 203 is made up of core i+1 to core N-1 as shown in Fig. 29, each of core i+1 to core N-1 extracts a code candidate area from the read image that is likely to contain a code as a repair code area, and sends a trigger signal to the inference processing unit 20j to execute inference processing. Although Fig. 29 shows one task being executed by two cores, one task may be executed by any one or more cores.

[0205] Specifically, when cores N-2 and N-1 constituting the second decoding unit 203 extract a repair code candidate area, they output a trigger signal (Trigger 1) to the inference processing unit 20j, causing the inference processing unit 20j to execute inference processing. Upon receiving the trigger signal (Trigger 1) sent from cores N-2 and N-1, the inference processing unit 20j inputs a partial image corresponding to the repair code candidate area extracted by cores N-2 and N-1 into a neural network and executes inference processing to generate a repaired image by repairing the partial image (Image Repair 1). The repaired image generated by Image Repair 1 is sent to cores N-2 and N-1. Based on the repaired image generated by Image Repair 1, cores N-2 and N-1 determine grid positions indicating the positions of each cell of the code and execute decoding processing of the repaired image based on the determined grid positions.

[0206] On the other hand, when core i+1 and core i+2 constituting the second decoding unit 203 extract a repair code candidate area, they output a trigger signal (trigger 2) to the inference processing unit 20j, causing the inference processing unit 20j to execute inference processing. Upon receiving the trigger signal (trigger 2) sent from core i+1 and core i+2, the inference processing unit 20j inputs the partial image corresponding to the repair code candidate area extracted by core i+1 and core i+2 into a neural network and executes inference processing to generate a repaired image by repairing the partial image (image restoration 2). The repaired image generated by image restoration 2 is sent to core i+1 and core i+2. Based on the repaired image generated by image restoration 2, core i+1 and core i+2 determine grid positions indicating the position of each cell of the code and execute decoding processing of the repaired image based on the determined grid positions.

[0207] Similarly, image restoration 3 is executed in response to a trigger signal (trigger 3) sent from core N-2 and core N-1, image restoration 4 is executed in response to a trigger signal (trigger 4) sent from core i+1 and core i+2, image restoration 5 is executed in response to a trigger signal (trigger 5) sent from core N-2 and core N-1, image restoration 6 is executed in response to a trigger signal (trigger 6) sent from core i+1 and core i+2, and image restoration 7 is executed in response to a trigger signal (trigger 7) sent from core N-2 and core N-1.

[0208] As shown in this time chart, the inference processing by the inference processing unit 20j and the first decoding processing by the first decoding processing unit 202 are executed in parallel. For example, while the first decoding processing unit 202 continues to perform code region extraction, grid positioning, and decoding processing, the inference processing by the inference processing unit 20j is executed once or multiple times. Also, while the first decoding processing unit 202 continues to perform code region extraction, grid positioning, and decoding processing, the second decoding processing unit 203 extracts a repair code region, outputs a trigger signal, positions a grid, and executes second decoding processing. In this way, the inference processing unit 20j only needs to specialize in repairing the partial image corresponding to the repair code candidate region, thereby increasing the inference processing speed.

[0209] At least a portion of the time during which the second decoding unit 20j is performing the second decoding process includes a pause period during which the inference processing unit 20j does not perform the inference processing. That is, as shown in FIG. 29, after the inference processing unit 20j completes image restoration 1, the second decoding unit 203 starts the second decoding process. However, a predetermined period is provided before the inference processing unit 20j starts the next image restoration 2. This predetermined period is a pause period during which the inference processing unit 20j does not perform image restoration. This pause period is a period during which the inference processing unit 20j does not perform inference processing. By providing this pause period, the load on the inference processing unit 20j is reduced, thereby suppressing heat generation in the AI ​​chip 23. Note that the processing time for grid positioning may be long or short depending on the image to be restored. While FIG. 29 illustrates an example in which the processing of each core is performed alternately, depending on the processing time for grid positioning, the processing of each core does not necessarily alternate. That is, trigger issuance and grid positioning may be performed in order starting from the core with the least processing time.

[0210] As described above, the setting unit 20e is configured to allow a user to select either "applying an image restoration filter" or "not applying an image restoration filter." Applying an image restoration filter means performing image restoration, and not applying an image restoration filter means not performing image restoration. Therefore, the setting unit 20e also corresponds to a part that sets whether or not the second decoding processing unit 203 performs the second decoding processing. Since this setting is performed by the user, the setting unit 20e is configured to be able to accept a user operation. The setting is not limited to applying or not applying an image restoration filter, but may also be, for example, applying or not applying an inference process (restoration process), or applying or not applying an AI process. The user operation may, for example, be the operation of a button or the like displayed on a user interface screen.

[0211] The optical information reading device 1 is operated after a tuning process is performed to determine in advance the settings of the imaging conditions such as exposure time and gain, and the image processing filter. For example, if the workpiece W is a label, the code is printed clearly, and if the appropriate conditions are set through the tuning process, a high reading rate can be achieved. However, for workpieces W made of metal or resin, even if the optimal conditions are set during the tuning process, reading may be difficult due to changes in the read image caused by weak ambient light, or the optical design and algorithm design may make it impossible to read due to extremely low contrast.

[0212] By performing the image restoration algorithm executed by the inference processing unit 20j of this example in parallel with the decoding process without image restoration, it is possible to improve the reading performance for a workpiece W made of metal, resin, etc. while maintaining conventional reading performance, and to increase reading stability. Furthermore, by applying the image restoration algorithm, it becomes possible to read a workpiece W that cannot be read using conventional optical designs and algorithms.

[0213] To give a specific example, in a process of reading a code printed on a sticker attached to a box in a factory, if there are no restrictions on the installation conditions of the optical information reading device 1 and optimal installation is possible, stable operation is possible even with conventional reading performance, so the image restoration filter can be disabled and takt time can be prioritized.

[0214] Also, in a certain factory, there is a process for reading codes that are clearly printed on metal, and although the optimal conditions are determined in the tuning process when setting up the reading, in actual operation there are cases where reading becomes unstable due to scratches on the metal surface, variations in the reading position and angle, changes in the brightness of the surrounding environment, etc. In such cases, applying an image restoration filter to improve the image quality makes it more resistant to variations in conditions such as those mentioned above, and improves reading performance.

[0215] In addition, some workpieces cannot be read using conventional algorithms, but in such cases, applying an image restoration filter can make them readable. For example, even if the code is severely damaged or the contrast is unclear, applying an image restoration filter can restore the contrast and scratches, making it possible to read the code.

[0216] The second decoding process executed when an image restoration filter is applied performs decoding on a restored image, and therefore often takes longer than the first decoding process, which performs decoding on a normal read image. Therefore, the first decoding processing unit 202 can execute the first decoding process in a shorter time than the time required for the second decoding process by the second decoding processing unit 203. In this example, based on this premise, the setting unit 20e is able to change the decoding timeout time, which is the time limit for the decoding process.

[0217] Specifically, when the second decoding process is set to be executed, the setting unit 20e is configured to be able to set a decoding timeout time longer than the time required to execute the second decoding process, and when the second decoding process is set not to be executed, the setting unit 20e is configured to be able to set a decoding timeout time shorter than the time required to execute the second decoding process. This allows a decoding timeout time longer than the time required to execute a typical second decoding process to be set during operation when the second decoding process is executed, thereby ensuring reliable decoding of the restored image. On the other hand, when the second decoding process is set not to be executed, only the first decoding process, which generally can be decoded in a shorter time than the second decoding process, is executed, so a short decoding timeout time corresponding to the first decoding process can be set. By changing the decoding timeout time in this manner, it is possible to set decoding timeout times corresponding to both the first decoding process and the second decoding process.

[0218] 29, the first decoding processing unit 202 and the second decoding processing unit 203 are separate units, but as shown in FIG. 30, the first decoding processing unit 202 and the second decoding processing unit 203 may be combined into a decoding processing unit 210. The decoding processing unit 210 performs a second decoding processing on the restored image in parallel with a first decoding processing on the scanned image generated by the camera 5. In this way, the parts that perform the decoding processing can be arbitrarily divided or integrated on the hardware. The following description will be given using an example in which the parts that perform the decoding processing are integrated into one unit, but this is not limiting and the same applies to cases in which the parts are divided into multiple units.

[0219] When the setting unit 20e sets the second decoding process not to be executed, the decoding processing unit 210 allocates processing resources used for the second decoding process to the first decoding process, thereby speeding up the first decoding process compared to when the second decoding process is set to be executed. Processing resources include, for example, memory usage areas and cores constituting a multi-core CPU. Specifically, when the setting unit 20e sets the second decoding process not to be executed, the number of cores responsible for the first decoding process is increased compared to when the setting unit 20e sets the second decoding process to be executed. Furthermore, when the setting unit 20e sets the second decoding process not to be executed, the memory area used for the first decoding process is expanded compared to when the setting unit 20e sets the second decoding process to be executed. Increasing the number of cores and expanding the memory area may be performed together, or either one of them may be performed alone. This maximizes the performance of the multi-core CPU to speed up the first decoding process.

[0220] Prior to image restoration, the decoding processor 210 extracts code candidate regions from the scanned image that are likely to contain a code. Specifically, when using a neural network to restore an image, if the image input to the neural network is large, the computation time increases. Furthermore, from the perspective of speeding up processing, it is desirable to minimize the number of decoding attempts. To address these issues, this example employs a heat map-based search method rather than a line search-based method to ensure reliable extraction of code candidate regions, even if it takes some time. For example, the decoding processor 210 quantifies the features of the code, generates a heat map in which the magnitude of the feature is assigned to each pixel value, and extracts code candidate regions from the heat map that are likely to contain a code. A specific example is a method of extracting feature portions (e.g., finder patterns) of a 2D code from areas that are relatively hot (large feature amounts) in the heat map. If multiple feature portions are acquired, they can be prioritized and extracted, and stored in the RAM 41, the storage unit 30, etc.

[0221] After the decoding processing unit 210 extracts a code candidate region, it outputs a partial image corresponding to the extracted code candidate region to the inference processing unit 20j. The inference processing unit 20j inputs the partial image corresponding to the code candidate region extracted by the decoding processing unit 210 into a neural network and executes inference processing to generate a restored image by restoring the partial image. This reduces the calculation time.

[0222] The decoding processing unit 210 may also be configured to extract a code candidate area in the scanned image that is large enough to contain the code to be read, as determined by the tuning process described above, and that is likely to contain the code. The extracted image is then reduced or enlarged to a predetermined size before being input to the neural network, and the restored image restored by the neural network is decoded. By enlarging or reducing the extracted image to a predetermined size and inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range, thereby stably achieving the image restoration effect of the neural network. Although the code size to be read may vary, the size of the code ultimately input to the neural network can be fixed, thereby maintaining a constant balance between high processing speed and ease of decoding.

[0223] The flowcharts of the first decoding process and the second decoding process will be described in detail below with reference to FIG. 31. In step SE1 after the start, the processor 20 causes the camera 5 to generate a read image and acquires the read image. The read image acquired in step SE1 is input to the decoding processing unit 210. In step SE2, the first decoding process is executed on the read image acquired in step SE1 without performing image restoration. If the first decoding process is successful, the process proceeds to step SE4 to execute termination processing, i.e., output processing of the decoded result. If the first decoding process fails in step SE2, the process returns to step SE2 and executes the first decoding process again. When a predetermined decoding timeout period has elapsed, the first decoding process for the read image is stopped.

[0224] On the other hand, in the route proceeding from step SE1 to step SE5, the decoding processing unit 210 extracts a repair code area from the read image. Specifically, it extracts a code candidate area that is large enough to include the code to be read, as set by the tuning process, and that is likely to contain the code in the read image. If multiple repair code areas are extracted at this time, they are temporarily saved as candidate areas R1, R2, .... An example of an extracted repair code area is shown in Figure 32. Then, it proceeds to step SE6, and the value of k is set to 1.

[0225] In step SE7, a resizing process is performed on the partial image corresponding to the candidate region Rk. The resizing process reduces or enlarges the partial image corresponding to the repair code region extracted in step SE5 to a predetermined size. This resizing process allows the size of the image to be input to the neural network, which will be described later, to be set to a predetermined size in advance. In the example shown in FIG. 32, the size of the partial image to be input to the neural network is 256 pixels x 256 pixels, but the size of the partial image is not limited to this and can be resized to any size taking into account processing speed, etc.

[0226] In step SE8, it is determined whether the black-and-white inversion setting is set to "both." In other words, image restoration is simply a mathematical mapping, and it may be difficult to achieve white-to-black, black-to-white, black-to-black, and white-to-white mappings using a single neural network. Therefore, black-and-white inversion processing is performed based on the properties to produce an image with black printed on a white background. Depending on how the neural network is trained, an image with white printed on a black background may also be possible. An example of black-and-white inversion processing is shown in FIG. 32. Details of the black-and-white inversion processing will be described later.

[0227] If the determination in step SE8 is YES and the black-and-white inversion setting is "both," the process proceeds to step SE9, where the resized image is input to the inference processing unit 20j and image restoration is performed. Thereafter, the process proceeds to step SE10, where the decoding processing unit 210 performs a second decoding process on the restored image. If the second decoding process is successful in step SE10, the process proceeds to step SE4 and executes termination processing. If the second decoding process fails in step SE10, the process proceeds to step SE11, where the resized image is subjected to black-and-white inversion processing. Thereafter, in step SE11, the image after the black-and-white inversion processing is input to the inference processing unit 20j and image restoration is performed (see FIG. 32). Next, the process proceeds to step SE13, where the decoding processing unit 210 performs a second decoding process on the restored image. If the second decoding process is successful in step SE13, the process proceeds to step SE4 and executes termination processing. If the second decoding process fails in step SE13, the process proceeds to step SE14. In step SE14, it is determined whether or not there are any more candidate regions extracted in step SE5, and if there are no more candidate regions, the process proceeds to step SE4 to execute the termination process.

[0228] On the other hand, if step SE8 returns NO and the black-and-white inversion setting is not set to "both," the process proceeds to step SE15, where it is determined whether the black-and-white inversion setting is set to "OFF." If step SE15 returns NO and the black-and-white inversion setting is not set to "OFF," the process proceeds to step SE16, where the image after the resizing process is inverted. Thereafter, in step SE17, the image after the black-and-white inversion process is input to the inference processing unit 20j, where image restoration is performed. Next, the process proceeds to step SE18, where the decoding processing unit 210 performs a second decoding process on the restored image. If the second decoding process is successful in step SE18, the process proceeds to step SE4, where termination processing is performed. If the second decoding process is unsuccessful in step SE18, the process proceeds to step SE14. In step SE14, it is determined whether or not there are still candidate areas. If there are no candidate areas, the process proceeds to step SE4, where termination processing is performed.

[0229] If it is determined in step SE14 that there are still candidate areas, the process proceeds to step SE19, where 1 is added to the value of k, and the process proceeds to step SE7. In step SE7, the value of k has increased by 1 since the previous flow, so a similar resizing process is performed on the partial image corresponding to the next candidate area Rk. Steps from SE8 onwards are as described above. In other words, if decoding is successful in either the first decoding process or the second decoding process, the decoded result is output at that point, and decoding processes are not performed on other candidate areas; instead, a scanned image of the next work W is obtained, and similar processes are performed.

[0230] The post-processing unit 201 shown in FIG. 32 performs a process of synthesizing a partial image that has not been restored after resizing with a partial image that has been restored by the inference processing unit 20j after resizing. In other words, when image restoration using a neural network is performed, even if the code before restoration is visible to the human eye, image restoration may not be successful, or cells may become round after image restoration. To address these issues, overlapping the partial image that has not been restored with the partial image that has been restored at an appropriate ratio may facilitate subsequent decoding. The overlap ratio between the partial image that has not been restored and the partial image that has been restored may be a predetermined fixed value or may be a non-fixed value. For example, it may be determined dynamically during reading or by optimization during pre-evaluation (tuning) based on the stability of restoration by the inference processing unit 20j. For example, in pre-evaluation, the ratio may be adjustable by the user using a scale bar or the like, allowing the user to adjust the ratio while the combined image is presented to the user.

[0231] (Bank switching function) In this embodiment, the parameter set storage unit 30c is configured to store parameter sets, each of which includes a set of parameters constituting the imaging conditions of the camera 5 and parameters constituting the processing conditions in the decoding processing unit 210. A parameter set can be called a bank. A plurality of banks are provided, each storing different parameters. For example, the parameter set storage unit 30c stores a plurality of imaging conditions and code conditions, including first imaging conditions and code conditions set by the tuning execution unit 20d and second imaging conditions and code conditions, as separate parameter sets.

[0232] The optical information reading device 1 is configured to be able to switch from one parameter set including first imaging conditions and code conditions to another parameter set including second imaging conditions and code conditions, or vice versa, among the multiple parameter sets stored in the parameter set storage unit 35c. The parameter set switching can be performed by the processor 20, a user, or a switching signal from an external control device such as the PLC 101. The user can switch parameter sets by, for example, operating a parameter set switching unit incorporated in the user interface. By enabling the parameter set switching unit, the parameter set in that bank is used during operation of the optical information reading device 1, and by disabling the parameter set switching unit, the parameter set in that bank is not used during operation of the optical information reading device 1. In other words, the parameter set switching unit is used to switch from one parameter set to another.

[0233] In this example, when the decoding processing unit 210 is operating using one parameter set and the second decoding process on the restored image fails, it switches to another parameter set and performs image restoration and second decoding on a scanned image captured under different conditions. For example, when the decoding processing unit 210 detects a failure in the second decoding process, it switches to a parameter set other than the current parameter set and performs image restoration and second decoding. If it detects that the second decoding process also fails with this parameter set, it switches to a parameter set not previously used and performs image restoration and second decoding. Switching parameter sets may change not only the imaging conditions of the camera 5 but also the code size to be read. In this case, the size used to extract code candidate areas where a code is likely to exist also changes. As a result, the reduction or enlargement ratio of the image input to the neural network also changes, changing the results of image restoration.

[0234] (Black and white inversion supported) Next, we will explain how to deal with black-and-white inversion of codes with reference to Figures 33 and 34. Figure 33 is a diagram illustrating how to deal with black-and-white inversion of codes using two neural networks, namely, one trained with a code printed in black on a white background and the other trained with a code printed in white on a black background. Figure 33A shows a method in which a scanned image (the image at the left) is input to a neural network trained with a code printed in black on a white background and a neural network trained with a code printed in white on a black background, and the decoded results are output from each neural network. This method is called a parallel execution method. Since the scanned image is an image of a code printed in black on a white background, decoding is successful when the image is restored using the neural network trained with the code printed in black on a white background, but decoding fails when the image is restored using the neural network trained with the code printed in white on a black background.

[0235] In FIG. 33B, the scanned image (the image on the left) is first input only to a neural network trained with a code printed in white on a black background, and an attempt is made to decode the restored image. If the decoding process fails, the scanned image is input to a neural network trained with a code printed in black on a white background to generate a restored image. This method uses a different network after a failure.

[0236] FIG. 33C shows a method for selecting a neural network to input a scanned image based on user settings. The case shown on the left is when the scanned image is set to be an image of a code printed in black on a white background. In this case, the network selection module selects a neural network trained with a code printed in black on a white background. The scanned image (the image at the left) is then input to the neural network trained with a code printed in black on a white background. On the other hand, the case shown on the right is when the scanned image is set to be an image of a code printed in white on a black background. In this case, the network selection module selects a neural network trained with a code printed in white on a black background. The scanned image (the image at the left) is then input to the neural network trained with a code printed in white on a black background.

[0237] FIG. 34 is a diagram illustrating how to deal with black-and-white inversion of a code using only one neural network. FIG. 34A shows how a black-and-white inversion image is generated by inverting the scanned image (the image on the left). An image without black-and-white inversion and an image with black-and-white inversion are input to a neural network trained with a code printed in black on a white background. Because the scanned image is an image of a code printed in white on a black background, decoding fails when using a restored image generated by a neural network trained with a code printed in black on a white background. However, decoding is successful when using an image after black-and-white inversion.

[0238] In the example shown at the top of FIG. 34B, the scanned image (the image on the left) is an image of a code printed in white on a black background, so when it is input to a neural network trained with a code printed in black on a white background, the decoding process fails. After that, when an image with black and white inversion is generated and input to a neural network trained with a code printed in black on a white background, the decoding process is successful.

[0239] In the example shown at the bottom of FIG.34B, the scanned image (the image on the left) is an image of a code printed in black on a white background, so when it is input to a neural network trained with a code printed in white on a black background, the decoding process fails. After that, when an image with black and white inversion is generated and input to a neural network trained with a code printed in white on a black background, the decoding process is successful.

[0240] FIG. 34C shows a method for determining whether to perform black-and-white inversion processing based on user settings. The upper case shows a case where the scanned image is set to be an image of a code printed in black on a white background. In this case, the black-and-white inversion processing module does not perform black-and-white inversion processing, and the image is input directly to a neural network trained with a code printed in black on a white background. On the other hand, the lower case shows a case where the scanned image is set to be an image of a code printed in white on a black background. In this case, the black-and-white inversion processing module performs black-and-white inversion processing. The black-and-white inverted image is then input to a neural network trained with a code printed in black on a white background.

[0241] (Relationship between contrast and matching level) The matching level is used, for example, as a score during tuning, or to manage print quality / reading performance during operation. For example, the matching level is defined as a positive integer value between 0 and 100, and a level of 50 or higher can be designed to achieve a 100% read rate when a read rate test is conducted. Also, a matching level of 0 means that the code cannot be read, and a matching level of 1 is the lowest value at which the code can be read.

[0242] 35 is a graph showing the relationship between contrast and matching level, where the solid line shows the relationship between contrast and matching level of an image (restored image) restored by the inference processing unit 20j, and the dashed line shows the relationship between contrast and matching level of an image (read image) that has not been restored by the inference processing unit 20j. As is clear from this graph, the matching level of the restored image is generally higher than that of the read image, but when the image quality deteriorates beyond a certain level, the matching level drops sharply.

[0243] (Matching level calculation process) A matching level is set for each algorithm, and typically one algorithm is used for each code type. However, in this embodiment, an image restoration function is implemented, and two algorithms (one with image restoration and one without image restoration) are executed for each code type. The result of the algorithm that first successfully decodes the image is output. Therefore, if the two algorithms alternately succeed in decoding, the matching level value may become unstable. Furthermore, since image restoration makes the image easier to read, conventional methods of calculating the matching level tend to result in a higher matching level than existing methods. In response to these issues, this embodiment calculates a weighted sum of the matching levels of the two algorithms and adjusts the matching level value. A specific example will be described below based on the flowchart shown in FIG. 36.

[0244] In step SF1 after the start, the read image is input to the decoding processing unit 210. In step SF2, the decoding processing unit 210 executes the first decoding processing without image restoration. In step SF3, a matching level A (MLA) is calculated by the first decoding processing. Meanwhile, in step SF4, the read image is input to the inference processing unit 20j and image restoration processing is executed. Thereafter, the process proceeds to step SF5, where the decoding processing unit 210 executes the second decoding processing on the restored image. In step SF6, a matching level B (MLB) is calculated by the second decoding processing. Note that if decoding is not successful in step SF2, MLA=0, and if decoding is not successful in step SF5, MLB=0.

[0245] In step SF7, the MLA and MLB values ​​are adjusted using, for example, an averaging function, etc. Furthermore, regardless of whether decoding is successful in step SF2 or SF5, a process of weighting the MLA value is performed.

[0246] (Example of user interface) FIG. 37 is a diagram showing an example of a user interface 300 displayed on the display unit 6. The processor 20 generates the user interface 300 and displays it on the display unit 6. The user interface 300 is provided with a first image display area 301 in which an image currently captured by the camera 5 (a live view image) is displayed, a second image display area 302 in which a read image is displayed, and a third display area 303 in which a restored image restored by the inference processing unit 20j is displayed. This allows the read image and the restored image to be presented to the user in a form that allows them to compare them. In addition, the user can understand what the image to be decoded or what the image after the decoding process is performed looks like.

[0247] (Example of an image sensor with an AI chip) FIG. 38 shows an example in which an imaging element 5a with an AI chip is used, and the imaging element 5a of the camera 5 can be configured as shown in each figure. FIG. 38A shows a packaged imaging element 5a equipped with an AI chip that performs image restoration. After the scanned image is restored, it is output to the processor 20. FIG. 38B shows a packaged imaging element 5a equipped with an AI chip that extracts a repair code area and performs image restoration. After extracting a repair code area from the scanned image, it repairs a partial image corresponding to that area and outputs it to the processor 20. FIG. 38C shows a packaged imaging element 5a equipped with an AI chip that extracts a repair code area. After extracting a repair code area from the scanned image, it outputs a partial image corresponding to that area to the processor 20.

[0248] In the example shown in FIG. 38A, the scanned image acquired by the image sensor 5a may be output to the processor 20, and the image restoration may be performed by an AI chip, and the restored image may be output to the processor 20. The same applies to the examples shown in FIGS. 38B and 38C.

[0249] (Polarizing filter attachment 3) In this embodiment, as shown in Fig. 2, the device includes a first illumination unit 4a and a second illumination unit 4b, each consisting of a plurality of light-emitting diodes, as light sources that generate illumination light for illuminating at least the code. The polarizing filter attachment 3 includes a first polarizing plate 3a that passes light of a first polarization component of the light generated by the light-emitting diodes that constitute the second illumination unit 4b, and a second polarizing plate 3b that passes light of a second polarization component that is approximately perpendicular to the first polarization component. The first polarizing plate 3a and the second polarizing plate 3b can be provided in the shaded areas in Fig. 2. For example, the first polarizing plate 3a is arranged to cover the second illumination unit 4b, and the second polarizing plate 3b is arranged to cover the optical system 5b of the camera 5 from the light incident side.

[0250] Therefore, the camera 5 receives light that passes through the first polarizer 3a and is reflected from the surface (code) of the workpiece W via the second polarizer 3b. This removes the specular reflection component of the workpiece W, and the camera 5 generates a read image with lower contrast than when the light does not pass through the first polarizer 3a and the second polarizer 3b. This low-contrast read image is acquired by the processor 20. The processor 20 inputs the low-contrast read image into a neural network, converting it into a restored image with higher contrast than before input, and performs a decoding process on the restored image.

[0251] In other words, by using polarizing plates 3a and 3b, the specular reflection component of the workpiece W is removed, making it possible to obtain a read image in which the influence of the specular reflection component is reduced. For example, polarizing plates 3a and 3b are suitable for images with a large specular reflection component, such as when capturing an image of a metal workpiece. However, the read image may become dark and the contrast may decrease due to a decrease in the amount of light. In such cases, reading performance can be improved by converting the image into a high-contrast restored image using neural network inference processing.

[0252] The above-described embodiments are merely examples in all respects and should not be construed as limiting. Furthermore, all modifications and variations within the scope of the claims are within the scope of the present invention. [Industrial Applicability]

[0253] As described above, the optical information reading device according to the present invention can be used, for example, to read a code attached to a workpiece. [Explanation of symbols]

[0254] 1. Stationary optical information reader 1A Handheld optical information reader 5. Camera 20 processors 20f Decode processing section 20j Inference processing section (image restoration section) 21~24 1st~4th general-purpose cores 25 dedicated cores (cores dedicated to inference processing) 40 ROM 40d Neural network memory section 41 RAM

Claims

1. An optical information reading device that reads a code attached to a workpiece, a camera that photographs the code and generates a scanned image; a memory unit for storing the structure and parameters of a neural network for estimating an ideal image corresponding to the scanned image generated by the camera; a processor having an inference processing unit that inputs a read image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference processing to generate an ideal image corresponding to the read image, and a decoding processing unit that executes a first decoding processing to decode the read image generated by the camera and a second decoding processing to decode the ideal image generated by the inference processing unit, The processor is configured to execute the inference processing by the inference processing unit and the first decoding processing by the decoding processing unit in parallel, and to execute the second decoding processing by the decoding processing unit after the inference processing by the inference processing unit is completed.

2. 2. The optical information reading device according to claim 1, The decoding processing unit includes a core that executes the first decoding process and the second decoding process in parallel in separate threads.

3. 2. The optical information reading device according to claim 1, The optical information reading device, wherein the decoding processing unit is configured with multiple cores capable of executing the first decoding process and the second decoding process using different cores.

4. 4. The optical information reading device according to claim 1, the decoding processing unit has a general-purpose core that executes an extraction process of extracting a code candidate area in which a code is likely to exist from the read image generated by the camera, and the first decoding process of decoding a partial image extracted from the read image by the extraction process, The inference processing unit is an optical information reading device having a core dedicated to inference processing that inputs a partial image extracted from a read image by extraction processing and performs inference processing to generate an ideal image corresponding to the read image.

5. 5. The optical information reading device according to claim 4, The general-purpose core executes a second decoding process to decode an ideal image generated by the inference process performed by the dedicated inference process core.

6. 6. The optical information reading device according to claim 4, The general-purpose core of the optical information reading device has a first core that performs the extraction process, a second core that performs the first decoding process, and a third core that performs the second decoding process.

7. 7. The optical information reading device according to claim 4, An optical information reading device in which, when an ideal image is generated by inference processing by the inference processing-dedicated core, the general-purpose core executes the second decoding process to decode the ideal image in priority to the first decoding process.

8. 8. The optical information reading device according to claim 5, the dedicated inference processing core, when a plurality of partial images are extracted as a result of the extraction processing by the general-purpose core, executes inference processing on the plurality of partial images in a predetermined order; When a plurality of partial images are extracted as a result of the extraction process, the general-purpose core executes the first decoding process on the plurality of partial images in a predetermined order.

9. 9. The optical information reading device according to claim 8, The optical information reading device is configured such that, if the decoding is successful in the first decoding process, the processor terminates the inference process by the inference processing unit.

10. 10. The optical information reading device according to claim 4, the general-purpose core executes a first extraction process for extracting the code candidate area under a first predetermined condition, a second extraction process for extracting the code candidate area under a second predetermined condition that enables extraction with higher accuracy than the first predetermined condition, and a first decoding process for decoding a partial image extracted from the read image by the first extraction process, The dedicated inference processing core receives the partial image extracted from the read image by the second extraction processing, and executes inference processing to generate an ideal image corresponding to the read image.

11. 11. The optical information reading device according to claim 4, The optical information reading device includes a memory in which a partial image extracted from the read image by the extraction process is stored and which is accessible by both the general-purpose core and the dedicated inference processing core.

12. 12. The optical information reading device according to claim 4, the general-purpose core and the inference processing dedicated core are mounted on the same substrate, The inference processing dedicated core is an optical information reader configured with a dedicated core dedicated to inference processing using a neural network.

13. 13. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a first decoding processing unit that executes a first decoding process on the scanned image generated by the camera; a second decoding processing unit that executes a second decoding process on the restored image generated by the image restoration unit, the second decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image, and sends a trigger signal to the image restoration unit to cause it to execute an inference process; when receiving the trigger signal sent from the second decoding processing unit, the image restoration unit inputs a partial image corresponding to the code candidate region extracted by the second decoding processing unit into the neural network, and executes an inference process to generate a restored image by restoring the partial image; The second decoding processing unit determines grid positions indicating the positions of each cell of the code based on the restored image, and performs decoding processing of the restored image based on the determined grid positions.

14. 14. The optical information reading device according to claim 13, An optical information reading device in which the inference processing by the image restoration unit and the first decoding processing by the first decoding processing unit are executed in parallel, and at least a portion of the period during which the second decoding processing unit is executing the second decoding processing has a pause period during which the inference processing by the image restoration unit is not executed.

15. 13. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a first decoding processing unit that executes a first decoding process on the scanned image generated by the camera; a second decoding processing unit that executes a second decoding process on the restored image generated by the image restoration unit; a setting unit that sets whether or not the second decoding process is to be executed, the first decoding processing unit is capable of executing the first decoding process in a shorter time than a time required for the second decoding process; The setting unit is configured to set a decode timeout period longer than the time required to execute the second decoding process when the second decoding process is set to be executed, and to set a decode timeout period shorter than the time required to execute the second decoding process when the second decoding process is set not to be executed.

16. 13. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a decoding processing unit that performs a first decoding process on the read image generated by the camera, and a second decoding process on the restored image generated by the image restoration unit in parallel with the first decoding process; a setting unit that sets whether or not the second decoding process is to be executed, When the setting unit is set not to execute the second decoding process, the decoding processing unit allocates the processing resources used for the second decoding process to the first decoding process, thereby speeding up the first decoding process compared to when the setting unit is set to execute the second decoding process.

17. 13. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a decoding processing unit that executes a decoding process on the restored image generated by the image restoration unit, the decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image, the image restoration unit inputs a partial image corresponding to the code candidate region extracted by the decoding processing unit into the neural network, and executes an inference process to generate a restored image by restoring the partial image; The decoding processing unit determines grid positions indicating the positions of each cell of the code based on the restored image, and decodes the restored image based on the determined grid positions.

18. 13. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a decoding processing unit that executes a decoding process on the restored image generated by the image restoration unit; a tuning execution unit that repeats the image capturing and decoding processes by changing the image capturing conditions of the camera and the decoding conditions of the decoding process, and determines optimal image capturing and decoding conditions based on a matching level that indicates the ease of reading the code calculated under each image capturing and decoding condition, and executes a tuning process that sets the size of the code to be read, The decoding processing unit An optical information reading device that extracts a code candidate area from the read image that is large enough to include the code to be read, as set by the tuning process, and that is likely to contain a code, reduces or enlarges the extracted image to a predetermined size, and then inputs it into the neural network, and decodes the restored image restored by the neural network.

19. 13. The optical information reading device according to claim 1, a plurality of light sources for generating illumination light for illuminating at least the code; a first polarizing plate that transmits light of a first polarization component out of the light generated by the light source; a second polarizing plate that transmits light of a second polarized component that is substantially orthogonal to the first polarized component; The camera receives light that passes through the first polarizing plate and is reflected from the code via the second polarizing plate, and removes the regular reflection component of the work, thereby generating a read image with lower contrast than when the first polarizing plate and the second polarizing plate are not used, The processor inputs the low-contrast read image into the neural network, converts it into a restored image with higher contrast than before input, and performs a decoding process on the restored image.

20. 13. The optical information reading device according to claim 1, an image restoration unit that inputs the scanned image generated by the camera into a neural network configured with the structure and parameters stored in the storage unit and executes an inference process to generate a restored image by restoring the scanned image; a decoding processing unit that executes a decoding process on the restored image generated by the image restoration unit; a tuning execution unit that is capable of changing the imaging conditions of the camera, repeating imaging and decoding processes, and setting a plurality of imaging conditions and code conditions for the code to be read based on a matching level that indicates the ease of reading the code calculated under each imaging condition, the storage unit stores a plurality of imaging conditions and code conditions including first imaging conditions and code conditions and second imaging conditions and code conditions set by the tuning execution unit; The decoding processing unit inputs a read image generated under the first imaging condition and code condition out of the multiple reading conditions stored in the memory unit into the neural network to generate a repaired image, and if the decoding process of the repaired image fails, inputs a read image generated under the second imaging condition and code condition into the neural network to generate a repaired image, and performs decoding of the repaired image.

Images