Image processing device, image processing method, and image processing program
The image processing device employs a deep learning model trained with Semantic Segmentation images to improve code decoding accuracy by minimizing the impact of shooting conditions, thus stabilizing the reading rate for code images.
Patent Information
- Application Number
- PCT/JP2024/017886
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2024-05-15
- Publication Date
- 2024-12-05
AI Technical Summary
Existing image processing technologies face challenges in maintaining a consistent reading rate for codes, such as barcodes and QR codes, due to varying shooting conditions, which affect the quality of the code image and hinder accurate decoding.
An image processing device and method utilizing a deep learning model that has undergone machine learning with Semantic Segmentation images as learning data, capable of outputting a binary image from a code image, thereby reducing the impact of shooting conditions and improving decoding accuracy.
The solution effectively suppresses the deterioration of the reading rate caused by shooting conditions, enhancing the reliability of code decoding by producing binary images less affected by imaging environment variations.
Smart Images

Figure JP2024017886_05122024_PF_FP_ABST
Abstract
Description
Image processing device, image processing method, and image processing program
[0001] The present disclosure relates to an image processing device, an image processing method, and an image processing program.
[0002] In recent years, products have been managed by attaching codes, such as one-dimensional barcodes or two-dimensional codes, to the surface of the product and associating information about the product with an identification code obtained by decoding the code. When reading a code attached to the surface of an product, the quality of the code image (hereinafter referred to as a "code image") obtained by an imaging device varies depending on imaging conditions, such as lighting conditions, code printing quality, and the characteristics of the imaging device used to read the code. Therefore, even when the same code and the same binarization operation rule are used, the binary image obtained by binarizing the code image may vary depending on the imaging conditions. As a result, the read rate may decrease.
[0003] To address such problems, for example, the invention described in Patent Document 1 proposes the use of a learning model that repairs unreadable code into an easily readable code.
[0004] JP 2019-67278 A
[0005] However, the invention described in Patent Document 1 uses a general decoding process, i.e., a binarization process using a threshold value, making it difficult to suppress deterioration of the reading rate due to the shooting conditions. It is desirable to provide an image processing device, an image processing method, and an image processing program that can suppress deterioration of the reading rate due to the shooting conditions.
[0006] An image processing device according to a first aspect of the present disclosure includes a receiving unit, a learning model, and a decoding unit. The receiving unit is capable of receiving a code image obtained by capturing an image of a code on a workpiece. The learning model is a machine-learned model that uses the captured code image and two classes of Semantic Segmentation images corresponding to the code image as training data. When a code image or a processed image obtained by performing a predetermined process on the code image is input, the learning model is capable of outputting a binary image corresponding to the code image or the processed image. The decoding unit is capable of generating an identification code by decoding the binary image.
[0007] The image processing method according to the second aspect of the present disclosure includes the following three steps: (A1) Acquiring a code image obtained by capturing an image of a code on a workpiece; (A2) Inputting the code image or a processed image obtained by performing a predetermined process on the code image into a learning model that has undergone machine learning using the code image captured by capturing the image and two classes of SemanticSegmentation images corresponding to the code image as training data, thereby acquiring a binary image corresponding to the code image or the processed image from the learning model; and (A3) Generating an identification code by decoding the binary image.
[0008] An image processing program according to a third aspect of the present disclosure includes a learning model that has undergone machine learning using a code image obtained by imaging and two classes of SemanticSegmentation images corresponding to the code image as learning data, and that is capable of outputting a binary image corresponding to an input image when an input image is input. This image processing program is capable of causing a computer to perform the following two operations: (B1) acquiring a code image obtained by imaging a code on a workpiece; and (B2) inputting a code image or a processed image obtained by performing a predetermined process on the code image into the learning model as an input image, thereby acquiring an image corresponding to the code image or the processed image from the learning model as a binary image.
[0009] FIG. 1 is a diagram illustrating an example of a schematic configuration of an image processing device according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating an example of a schematic configuration of the image processing unit of FIG. 1. FIG. 3 is a diagram illustrating an example of a schematic configuration of the learning device of FIG. 1. FIG. 4 is a diagram illustrating an example of a processing procedure in the image processing device of FIG. 1. FIG. 5 is a diagram illustrating a modified example of the schematic configuration of the image processing unit of FIG. 2. FIG. 6 is a diagram illustrating a modified example of the schematic configuration of the learning device of FIG. 3. FIG. 7 is a diagram illustrating an example of a processing procedure in an image processing device including the image processing unit of FIG. 5. FIG. 8 is a diagram illustrating a modified example of the schematic configuration of the image processing device of FIG. 1. FIG. 9 is a diagram illustrating an example of a schematic configuration of the image processing unit of FIG. 8. FIG. 10 is a diagram illustrating an example of a schematic configuration of the learning device of FIG. 8. FIG. 11 is a diagram illustrating an example of a processing procedure in the image processing device of FIG. 8. FIG. 12 is a diagram illustrating an example of a processing procedure in the learning device of FIG. 8. FIG. 13 is a diagram illustrating a modified example of the schematic configuration of the image processing unit of FIG. 2. FIG. 14 is a diagram illustrating an example of a processing procedure in an image processing device including the image processing unit of FIG. 13. FIG. 15 is a diagram illustrating a modified example of the schematic configuration of the image processing unit of FIG. 2. Fig. 16 is a diagram showing an example of a processing procedure in an image processing device including the image processing unit of Fig. 15. Fig. 17 is a diagram showing a modified example of the schematic configuration of the image processing device of Fig. 1. Fig. 18 is a diagram showing an example of the schematic configuration of the image processing unit of Fig. 17. Fig. 19 is a diagram showing a modified example of the schematic configuration of the image processing unit of Fig. 17. Fig. 20 is a diagram showing a modified example of the schematic configuration of the image processing unit of Fig. 17. Fig. 21 is a diagram showing a modified example of the schematic configuration of the image processing unit of Fig. 17. Fig. 22 is a diagram showing a modified example of the schematic configuration of the image processing unit of Fig. 17.
[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be made in the following order: 1. Embodiment (FIGS. 1 to 4) 2. Modifications (FIGS. 5 to 22)
[0011] 1. Embodiment [Configuration] An image processing device 100 according to an embodiment of the present disclosure will be described. Fig. 1 illustrates a schematic configuration example of the image processing device 100. As shown in Fig. 1, the image processing device 100 includes, for example, a lens 110, an imaging unit 120, and an image processing unit 130. The image processing unit 130 corresponds to a specific example of an "image processing device" according to an embodiment of the present disclosure.
[0012] The lens 110 is configured to have one or more lenses, and is capable of guiding light from a subject (incident light) to the imaging unit 120 and forming an image on the light receiving surface of the imaging unit 120.
[0013] The imaging unit 120 is capable of periodically accumulating signal charges in response to light that is imaged on the light-receiving surface via the lens 110. The signal charges accumulated in the imaging unit 120 are transferred as pixel signals (image data) to a DSP circuit within the imaging unit 120. That is, the imaging unit 120 is capable of receiving image light (incident light) incident via the lens 110 and outputting pixel signals corresponding to the received image light (incident light) to the DSP circuit. The DSP circuit is a signal processing circuit that processes the pixel signals (image data). The imaging unit 120 is capable of temporarily storing images processed by the DSP circuit in a frame memory on a frame-by-frame basis. The imaging unit 120 is capable of outputting images read from the frame memory to the image processing unit 130 as RAW images.
[0014] The image capturing unit 120 may be capable of performing processing to reduce the dependency of the RAW image on the imaging environment in order to facilitate processing in the downstream image processing unit 130. In this case, the image capturing unit 120 is capable of performing, for example, demosaicing, linear matrix calculation, or gamma correction on the RAW image in accordance with the imaging environment. The image capturing unit 120 is capable of outputting the image obtained by the above processing to the image processing unit 130, for example.
[0015] In this embodiment, the subject is a code on a workpiece. Codes include both one-dimensional barcodes and two-dimensional codes. Examples of two-dimensional codes include QR Code (registered trademark), Micro QR Code (registered trademark), Data Matrix (Data code), Veri Code, Aztec Code, PDF417, and Maxi Code. Two-dimensional codes include stacked and matrix types, and in this embodiment, the code may be any of these two-dimensional codes. The code may be attached to the workpiece by directly printing or engraving it, or may be attached by printing it on a label and then affixing it to the workpiece.
[0016] Furthermore, in this embodiment, the image output from the imaging unit 120 (the image input to the image processing unit 130) is a code image obtained by imaging the code on the workpiece with the imaging unit 120. Hereinafter, this code image will be referred to as a code image Ic. When reading the code on the workpiece, the code image Ic may have variations in quality due to imaging conditions such as lighting conditions, the print quality of the code, and the characteristics of the imaging device 120. The code image Ic corresponds to a specific example of a "code image" according to an embodiment of the present disclosure.
[0017] 2 illustrates an example of a schematic configuration of the image processing unit 130. As illustrated in FIG. 2, the image processing unit 130 includes, for example, a reception unit 131, a storage unit 132, a control unit 133, a decoding unit 134, an output unit 135, and a communication unit 136. The reception unit 131, the storage unit 132, the control unit 133, the decoding unit 134, the output unit 135, and the communication unit 136 are connected to a bus 137, and are capable of transmitting and receiving data to and from each other via the bus 137. The reception unit 131 corresponds to a specific example of a "reception unit" according to an embodiment of the present disclosure. The decoding unit 134 corresponds to a specific example of a "decoding unit" according to an embodiment of the present disclosure.
[0018] The receiving unit 131 is an input interface that can receive the code image Ic from the imaging unit 120. The receiving unit 131 can output the received code image Ic to the control unit 133.
[0019] The storage unit 132 is configured, for example, with a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The storage unit 132 stores a deep learning model 132a and an image processing program 132b. The deep learning model 132a corresponds to a specific example of a "first learning model" according to an embodiment of the present disclosure. The deep learning model 132a and the image processing program 132b correspond to a specific example of an "image processing program" according to an embodiment of the present disclosure.
[0020] The deep learning model 132a is a network capable of estimating segmentation, such as a U-net. The deep learning model 132a is, for example, a SemanticSegmentation model capable of classifying each pixel, for example, a SemanticSegmentation model capable of classifying each pixel into two classes (e.g., foreground and background). The deep learning model 132a is a model that has undergone machine learning using, as training data, a code image obtained by capturing an image and a SemanticSegmentation image corresponding to the code image that has been subjected to two-class classification. The deep learning model 132a is the same model as the deep learning model 221 (described below) stored in the learning device 200, and is, for example, a copy of the deep learning model 221. When a code image Ic or a processed image obtained by performing a predetermined process on the code image Ic is input, the deep learning model 132a is capable of outputting a binary image Ib corresponding to the code image Ic or the processed image.
[0021] Image processing program 132b is a program describing a series of procedures for performing image processing in image processing unit 130. For example, when image processing program 132b is loaded into control unit 133, the series of procedures for performing image processing described in image processing program 132b are executed in control unit 133.
[0022] The control unit 133 includes, for example, a graphics processing unit (GPU) that performs image processing or a central processing unit (CPU) on which an image processing program 132b is loaded. The control unit 133 is capable of acquiring a code image Ic via the reception unit 131. The control unit 133 inputs the acquired code image Ic to the deep learning model 132a and acquires a binary image Ib in response to the input. The control unit 133 inputs the acquired binary image Ib to the decoding unit 134 and acquires an identification code Dcode in response to the input. The control unit 133 is capable of outputting the acquired identification code Dcode to the output unit 135. The control unit 133 is capable of acquiring a copy of the deep learning model 221 via the communication unit 136. The control unit 133 is capable of storing the acquired copy of the deep learning model 221 in the memory unit 132 as the deep learning model 132a. In other words, the control unit 133 is able to update the deep learning model 132a using the acquired copy of the deep learning model 221.
[0023] The decoding unit 134 is capable of generating the identification code Dcode by decoding the binary image Ib based on a decoding algorithm in accordance with the standard of the code on the workpiece. The decoding unit 134 is configured, for example, by an IC capable of decoding the binary image Ib.
[0024] The output unit 135 is an output interface capable of converting the identification code Dcode into output data Dout in a predetermined data format and outputting it to the outside. The communication unit 136 is a communication interface capable of communicating with the learning device 200 via a predetermined network. The communication unit 136 is capable of receiving a copy of the deep learning model 221 from the learning device 200. The communication unit 136 is capable of outputting the received copy of the deep learning model 221 to the control unit 133. The "predetermined network" includes, for example, the Internet, a cloud network, or a network specific to an operator.
[0025] Fig. 3 shows an example of the general configuration of learning device 200. As shown in Fig. 3, learning device 200 includes, for example, a communication unit 210, a storage unit 220, and a control unit 230. Communication unit 210, storage unit 220, and control unit 230 are connected to a bus 240, and are capable of transmitting and receiving data to and from each other via bus 240.
[0026] The communication unit 210 is a communication interface capable of communicating with the image processing device 100 via the above-mentioned "predetermined network." The communication unit 210 is capable of transmitting a copy of the deep learning model 221 input from the control unit 230 to the image processing device 100.
[0027] The storage unit 220 is configured, for example, with a volatile memory such as a DRAM, or a non-volatile memory such as an EEPROM or a flash memory. The storage unit 220 stores a deep learning model 221, a raw code image 222, a semantic segmentation image 223, and a learning program 224. The raw code image 222 corresponds to a specific example of a "first code image" according to an embodiment of the present disclosure. The semantic segmentation image 223 corresponds to a specific example of a "semantic segmentation image" according to an embodiment of the present disclosure.
[0028] The RAW code image 222 is a RAW image obtained by imaging, and is a code image obtained by imaging a code on a workpiece. The SemanticSegmentation image 223 is a SemanticSegmentation image corresponding to the RAW code image 222. In the SemanticSegmentation image 223, each pixel is classified into two classes (e.g., foreground and background).
[0029] The deep learning model 221 is a network capable of estimating segmentation, such as a U-net. The deep learning model 221 is, for example, a SemanticSegmentation model capable of classifying each pixel, for example, a SemanticSegmentation model capable of classifying each pixel into two classes (e.g., foreground and background). The deep learning model 221 is a model that has undergone machine learning using a raw code image 222 obtained by imaging and a SemanticSegmentation image 223 corresponding to the raw code image 222 as training data. When a code image Ic is input, the deep learning model 221 is capable of outputting a binary image Ib corresponding to the code image Ic.
[0030] The learning program 224 is a program in which a series of procedures for performing machine learning on the deep learning model 221 in the learning device 200 is described. For example, when the learning program 224 is loaded into the control unit 230, the series of procedures for performing machine learning described in the learning program 224 is executed in the control unit 230.
[0031] The control unit 230 is configured to include, for example, a GPU that performs machine learning on the deep learning model 221, or a CPU on which a learning program 224 is loaded. The control unit 230 is capable of performing machine learning on the deep learning model 221 using the RAW code image 222 and the SemanticSegmentation image 223 as learning data. The control unit 230 is capable of outputting a copy of the trained deep learning model 221 to the communication unit 210.
[0032] [Image Processing] Next, image processing in the image processing device 100 will be described.
[0033] 4 shows an example of a processing procedure for image processing in the image processing device 100. First, the control unit 133 acquires a code image Ic from the imaging unit 120 (step S101). Next, the control unit 133 inputs the acquired code image Ic to the deep learning model 132a (step S102). When the code image Ic is input, the deep learning model 132a outputs a binary image Ib corresponding to the code image Ic. The control unit 133 acquires the binary image Ib from the deep learning model 132a (step S103).
[0034] Next, the control unit 133 inputs the binary image Ib to the decoding unit 134. When the decoding unit 134 receives the binary image Ib, it generates an identification code Dcode by decoding the binary image Ib based on a decoding algorithm in accordance with the standard of the code on the work (step S104). The decoding unit 134 outputs the generated identification code Dcode to the control unit 133. The control unit 133 acquires the identification code Dcode. The control unit 133 outputs the acquired identification code Dcode to the output unit 135. When the identification code Dcode is input from the control unit 133, the output unit 135 converts the input identification code Dcode into output data Dout in a predetermined data format and outputs the output data Dout to the external device 300 (step S105). The external device 300 performs predetermined information processing using the output data Dout input from the image processing device 100. In this manner, image processing in the image processing device 100 is performed.
[0035] [Effects] Next, the effects of the image processing device 100 will be described.
[0036] In this embodiment, a code image Ic obtained by the imaging unit 120 is input to a deep learning model 132a that has undergone machine learning using the RAW code image 222 and the SemanticSegmentation image 223 as training data, and thereby a binary image Ib corresponding to the code image Ic is obtained. The obtained binary image Ib is then decoded to generate an identification code Dcode. This makes it possible to obtain a binary image Ib that is less affected by the imaging conditions than when a general decoding process, i.e., a binarization process using a threshold value, is performed. As a result, it is possible to suppress a deterioration in the reading rate due to the imaging conditions.
[0037] 2. Modifications Next, a description will be given of modifications of the image processing device 100 according to the above embodiment. In the following, common components will be assigned common reference numerals, and descriptions of the common components will be omitted as appropriate.
[0038] [Variation A] In the above embodiment, the image processing unit 130 may further include a storage unit 138 in which a deep learning model 138a is stored, as shown in FIG. 5. The deep learning model 138a corresponds to a specific example of a "second learning model" according to an embodiment of the present disclosure. Furthermore, in the above embodiment, the learning device 200 may further include a storage unit 250 in which a deep learning model 251, RAW code images 252 and 253, and a learning program 254 are stored, as shown in FIG. 6.
[0039] The storage unit 138 is configured, for example, by a volatile memory such as a DRAM or a non-volatile memory such as an EEPROM or a flash memory. The deep learning model 138a is, for example, a neural network such as an autoencoder. The deep learning model 138a is a model that has undergone machine learning using, for example, code images obtained by imaging and defect images in which defects have been added to the code images as training data. The deep learning model 138a is the same model as the deep learning model 251 stored in the learning device 200, and is, for example, a copy of the deep learning model 251. When a code image Ic is input, the deep learning model 138a is capable of outputting, as a processed image Id, an image in which defects contained in the code image Ic have been repaired.
[0040] The control unit 133 is capable of acquiring a code image Ic via the reception unit 131. The control unit 133 inputs the acquired code image Ic to the deep learning model 138a and, in response to the input, acquires an image in which a defect in the code image Ic has been repaired as a processed image Id. The control unit 133 inputs the acquired processed image Id to the deep learning model 132a and, in response to the input, acquires a binary image Ib. The control unit 133 inputs the acquired binary image Ib to the decoding unit 134 and, in response to the input, acquires an identification code Dcode. The control unit 133 is capable of outputting the acquired identification code Dcode to the output unit 135.
[0041] The control unit 133 is capable of acquiring copies of the deep learning models 221 and 251 via the communication unit 136. The control unit 133 is capable of storing the acquired copy of the deep learning model 221 in the storage unit 132 as the deep learning model 132a. That is, the control unit 133 is capable of updating the deep learning model 132a using the acquired copy of the deep learning model 221. The control unit 133 is capable of storing the acquired copy of the deep learning model 251 in the storage unit 138 as the deep learning model 138a. That is, the control unit 133 is capable of updating the deep learning model 138a using the acquired copy of the deep learning model 251.
[0042] The communication unit 136 is capable of receiving copies of the deep learning models 221 and 251 from the learning device 200. The communication unit 136 is capable of outputting the received copies of the deep learning models 221 and 251 to the control unit 133.
[0043] The storage unit 250 is configured, for example, with a volatile memory such as a DRAM, or a non-volatile memory such as an EEPROM or a flash memory. The RAW code image 252 is a RAW image obtained by imaging a code on a workpiece. The RAW code image 253 is a defect image in which a defect (defective portion df) is added to the RAW code image 252.
[0044] The deep learning model 251 is, for example, a neural network such as an autoencoder. The deep learning model 251 is a model that has undergone machine learning using, as learning data, a raw code image 252 obtained by imaging and a raw code image 253 that is a defect image in which a defect (defective portion df) is added to the raw code image 252. When a code image Ic is input, the deep learning model 251 is capable of outputting, as a processed image Id, an image in which the defect in the code image Ic has been repaired.
[0045] The learning program 254 is a program in which a series of procedures for performing machine learning on the deep learning model 251 in the learning device 200 is described. For example, when the learning program 254 is loaded into the control unit 230, the series of procedures for performing machine learning described in the learning program 254 is executed in the control unit 230.
[0046] The control unit 230 is configured to include, for example, a GPU that performs machine learning on the deep learning model 251, or a CPU on which the learning program 251 is loaded. The control unit 230 is capable of performing machine learning on the deep learning model 251 using RAW code images 252 and 253 as learning data. The control unit 230 is capable of outputting a copy of the trained deep learning model 251 to the communication unit 210.
[0047] Next, image processing in the image processing device 100 according to this modified example will be described.
[0048] FIG. 7 illustrates an example of the image processing procedure in the image processing device 100 according to this modification. First, the control unit 133 acquires a code image Ic from the imaging unit 120 (step S201). Next, the control unit 133 inputs the acquired code image Ic to the deep learning model 138a (step S202). When the code image Ic is input, the deep learning model 138a outputs an image in which defects in the code image Ic have been repaired as a processed image Id. The control unit 133 acquires the processed image Id from the deep learning model 138a (step S203). The control unit 133 inputs the acquired processed image Id to the deep learning model 132a (step S204). When the processed image Id is input, the deep learning model 132a outputs a binary image Ib corresponding to the processed image Id. The control unit 133 acquires the binary image Ib from the deep learning model 132a (step S205).
[0049] Next, the control unit 133 inputs the binary image Ib to the decoding unit 134. When the decoding unit 134 receives the binary image Ib, it generates an identification code Dcode by decoding the binary image Ib based on a decoding algorithm for the standard of the code on the work (step S206). The decoding unit 134 outputs the generated identification code Dcode to the control unit 133. The control unit 133 acquires the identification code Dcode. The control unit 133 outputs the acquired identification code Dcode to the output unit 135. When the identification code Dcode is input from the control unit 133, the output unit 135 converts the input identification code Dcode into output data Dout in a predetermined data format and outputs the output data Dout to the external device 300 (step S207). The external device 300 performs predetermined information processing using the output data Dout input from the image processing device 100. In this manner, image processing in the image processing device 100 according to this modification is performed.
[0050] Next, the effects of the image processing device 100 according to this modification will be described.
[0051] In this modification, a code image Ic obtained by the imaging unit 120 is input to a deep learning model 138a that has undergone machine learning using the raw code images 252 and 253 as training data, thereby obtaining an image (processed image Id) in which defects in the code image Ic have been repaired. Then, a binary image Ib is obtained by inputting the processed image Id to the deep learning model 132a. As a result, even if the code on the workpiece or the lens 110 is dirty, for example, it is possible to obtain a binary image Ib in which the influence of such imaging conditions is suppressed. As a result, it is possible to suppress a deterioration in the reading rate due to imaging conditions.
[0052] [Modification B] In the above embodiment and its modifications, the image processing unit 130 may be capable of outputting a control signal ctl instructing the imaging unit 120 to change the imaging conditions, as shown in Fig. 8, for example. In this modification, the control unit 133 corresponds to a specific example of an "imaging support unit" according to an embodiment of the present disclosure.
[0053] FIG. 9 shows an example of a schematic configuration of the image processing unit 130 according to this modified example. For example, as shown in FIG. 9 , the image processing unit 130 further includes a transmission unit 139. The transmission unit 139 is an output interface capable of outputting a control signal ctl to the imaging unit 120. If the decoding unit 134 is unable to generate the identification code Ib, the control unit 133 is capable of repeatedly outputting the control signal ctl to the transmission unit 139 while changing the imaging conditions for the imaging device 120 until the decoding unit 134 is able to generate the identification code Ib. When the control signal ctl is input from the image processing unit 130, the imaging unit 120 is capable of changing (setting) the imaging conditions in accordance with the input control signal ctl.
[0054] FIG. 10 illustrates a schematic configuration example of a learning device 200 according to this modification. For example, as shown in FIG. 10, in addition to a deep learning model 221 to a learning program 224, a raw code image 225 and a binary image 226 are further stored in the storage unit 220. The raw code image 225 and the binary image 226 are generated in conjunction with image processing by the image processing unit 130 according to this modification. The raw code image 225 is a code image Ic obtained by the imaging unit 120 when the imaging conditions are changed (set) in accordance with a control signal ctl. The binary image 226 is a binary image Ib output from the deep learning model 132a when the raw code image 225 is input to the deep learning model 132a. The deep learning model 221 corresponds to a specific example of a "master learning model that is backup data of a first learning model" according to an embodiment of the present disclosure.
[0055] The control unit 230 is capable of performing machine learning (additional learning) on the deep learning model 221 using the RAW code image 225 and the binary image 226 as learning data. The control unit 230 is capable of outputting a copy of the deep learning model 221 that has undergone additional learning to the communication unit 210. The communication unit 210 is capable of transmitting a copy of the deep learning model 221 that has undergone additional learning, input from the control unit 230, to the image processing device 100.
[0056] Next, image processing in the image processing device 100 according to this modified example will be described.
[0057] 11 shows an example of the processing procedure for image processing in the image processing device 100 according to this modification. First, the control unit 133 acquires a code image Ic from the imaging unit 120 (step S301). Next, the control unit 133 inputs the acquired code image Ic to the deep learning model 132a (step S302). When the code image Ic is input, the deep learning model 132a outputs a binary image Ib corresponding to the code image Ic. The control unit 133 acquires the binary image Ib from the deep learning model 132a (step S303).
[0058] At this time, the control unit 133 transmits the code image Ic and the binary image Ib to the learning device 200 via the communication unit 136 (step S304). Upon receiving the code image Ic and the binary image Ib from the image processing device 100, the learning device 200 stores the received code image Ic and the binary image Ib in the storage unit 220 as a RAW code image 225 and a binary image 226.
[0059] Next, the control unit 133 inputs the binary image Ib to the decoding unit 134. When the decoding unit 134 receives the binary image Ib, it decodes it based on a decoding algorithm specific to the code on the workpiece (step S305). The control unit 133 then determines whether the decoding of the binary image Ib was successful (step S306). If the decoding of the binary image Ib fails (step S306; N), i.e., if the decoding unit 134 is unable to generate the identification code Dcode from the binary image Ib, the control unit 133 outputs a control signal ctl to change the imaging conditions of the imaging unit 120 (step S307). The control unit 133 then executes steps S301 to S305 again using the code image Ic obtained by the imaging unit 120 under the changed imaging conditions.
[0060] On the other hand, if the decoding of the binary image Ib is successful (step S306; Y), that is, if the decoding unit 134 is able to generate the identification code Dcode from the binary image Ib, it outputs the generated identification code Dcode to the control unit 133. The control unit 133 acquires the identification code Dcode. The control unit 133 outputs the acquired identification code Dcode to the output unit 135. When the identification code Dcode is input from the control unit 133, the output unit 135 converts the input identification code Dcode into output data Dout in a predetermined data format and outputs it to the external device 300 (step S308). The external device 300 performs predetermined information processing using the output data Dout input from the image processing device 100.
[0061] The control unit 133 acquires a copy of the deep learning model 221 that has undergone additional training from the learning device 200. The control unit 133 updates the deep learning model 132a using the acquired copy of the deep learning model 221 (step S309). In other words, the control unit 133 replaces the deep learning model 132a with the copy of the deep learning model 221 that has undergone additional training. In this manner, image processing in the image processing device 100 is performed.
[0062] Next, additional learning in the learning device 200 according to this modification will be described.
[0063] 12 shows an example of a processing procedure for additional learning in the learning device 200 according to this modified example. First, the control unit 230 receives a code image Ic and a binary image Ib from the image processing device 100 via the communication unit 210 (step S401). The control unit 230 stores the received code image Ic and binary image Ib in the storage unit 220 as a RAW code image 225 and a binary image 226. Every time the control unit 230 receives a code image Ic and a binary image Ib, the control unit 230 stores the received code image Ic and binary image Ib in the storage unit 220 as the RAW code image 225 and the binary image 226.
[0064] The control unit 230 then performs machine learning (additional learning) on the deep learning model 221 using the RAW code image 225 and the binary image 226 stored in the storage unit 220 as learning data (step S402). The resulting trained deep learning model 221 corresponds to a specific example of a "new learning model" according to an embodiment of the present disclosure. The control unit 230 transmits a copy of the trained deep learning model 221 to the image processing device 100 via the communication unit 210 (step S403). Upon receiving the copy of the trained deep learning model 221 from the learning device 200, the image processing device 100 uses the received copy of the trained deep learning model 221 to update the deep learning model 132a. In other words, the image processing device 100 replaces the deep learning model 132a with the copy of the trained deep learning model 221. In this manner, the training in the learning device 200 is performed.
[0065] Next, the effects of the image processing device 100 according to this modification will be described.
[0066] In this modification, if the decoding unit 134 is unable to generate the identification code Dcode, the deep learning model 132a is replaced with a trained deep learning model 221 obtained by performing machine learning (additional learning) using as training data a raw code image 225 and a binary image 226 obtained while changing the imaging conditions. This makes it possible to obtain a binary image Ib that is less affected by the imaging conditions than when a general decoding process, i.e., a binarization process using a threshold, is performed. As a result, it is possible to suppress a deterioration in the reading rate due to the imaging conditions.
[0067] [Modification C] In the above embodiment and its modifications, the storage unit 132 may store, for example, a determination condition 132c as shown in Fig. 13. The determination condition 132c is information for determining whether or not the output data Dout can be output.
[0068] 14 shows an example of the processing procedure for image processing in the image processing device 100 according to this modification. First, the control unit 133 acquires a code image Ic from the imaging unit 120 (step S501). Next, the control unit 133 inputs the acquired code image Ic to the deep learning model 132a (step S502). When the code image Ic is input, the deep learning model 132a outputs a binary image Ib corresponding to the code image Ic. The control unit 133 acquires the binary image Ib from the deep learning model 132a (step S503).
[0069] Next, the control unit 133 inputs the binary image Ib to the decoding unit 134. When the decoding unit 134 receives the binary image Ib, it generates an identification code Dcode by decoding the binary image Ib based on a decoding algorithm in accordance with the standard of the code on the work (step S504). The decoding unit 134 outputs the generated identification code Dcode to the control unit 133. The control unit 133 acquires the identification code Dcode.
[0070] Next, the control unit 133 determines whether the identification code Dcode satisfies the judgment condition 132c (predetermined condition) (step S505). If the identification code Dcode does not satisfy the judgment condition 132c (predetermined condition) (step S505; N), the control unit 133 performs processing to indicate that the identification code Dcode was not correctly read. The control unit 133, for example, cancels output of the identification code Dcode to the output unit 135 (step S506). On the other hand, if the identification code Dcode satisfies the judgment condition 132c (predetermined condition) (step S505; Y), the control unit 133 outputs the identification code Dcode to the output unit 135 (step S507). When the identification code Dcode is input from the control unit 133, the output unit 135 converts the input identification code Dcode into output data Dout in a predetermined data format and outputs the output data Dout to the external device 300. The external device 300 performs predetermined information processing using the output data Dout input from the image processing device 100. In this manner, image processing in the image processing device 100 is performed.
[0071] In this modification, when the identification code Dcode does not satisfy the judgment condition 132c, a process is performed to indicate that the identification code Dcode is not correctly read data, and when the identification code Dcode satisfies the judgment condition 132c, the identification code Dcode is output to the output unit 135. This makes it possible to prevent the external device 300 from performing a process based on an incorrect identification code Dcode.
[0072] [Variation D] In the above embodiment and its variations, the storage unit 132 may store, for example, a determination condition 132d as shown in FIG. 15. The determination condition 132d is information for determining whether or not to output the output data Dout. Furthermore, in the above embodiment and its variations, the image processing unit 130 may further include, for example, an encoding unit 141 as shown in FIG. 15. The encoding unit 141 is capable of generating a binary image Ib' by encoding the identification code Dcode based on an encoding algorithm in accordance with the standard of the code on the work. The encoding unit 141 is configured, for example, with an IC capable of encoding the identification code Dcode. In this variation, the control unit 133 corresponds to a specific example of a "determination unit" according to an embodiment of the present disclosure.
[0073] 16 shows an example of the image processing procedure in the image processing device 100 according to this modification. First, the control unit 133 acquires a code image Ic from the imaging unit 120 (step S601). Next, the control unit 133 inputs the acquired code image Ic to the deep learning model 132a (step S602). When the code image Ic is input, the deep learning model 132a outputs a binary image Ib corresponding to the code image Ic. The control unit 133 acquires the binary image Ib from the deep learning model 132a (step S603).
[0074] Next, the control unit 133 inputs the binary image Ib to the decoding unit 134. When the decoding unit 134 receives the binary image Ib, it generates an identification code Dcode by decoding the binary image Ib based on a decoding algorithm in the standard of the code on the work (step S604). The decoding unit 134 outputs the generated identification code Dcode to the control unit 133. The control unit 133 acquires the identification code Dcode. Next, the control unit 133 outputs the identification code Dcode to the encoding unit 141. When the identification code Dcode is received, the encoding unit 141 generates a binary image Ib' by encoding the identification code Dcode based on an encoding algorithm in the standard of the code on the work (step S605).
[0075] Next, the control unit 133 calculates the matching rate between the binary image Ib and the binary image Ib' (step S606). The control unit 133 then determines whether the calculated matching rate satisfies the judgment condition 132d (predetermined condition) (step S607). If the matching rate does not satisfy the judgment condition 132d (predetermined condition) (step S607; N), the control unit 133 performs processing to indicate that the identification code Dcode was not correctly read. For example, the control unit 133 cancels output of the identification code Dcode to the output unit 135 (step S608). On the other hand, if the matching rate satisfies the judgment condition 132d (predetermined condition) (step S607; Y), the control unit 133 outputs the identification code Dcode to the output unit 135 (step S609). When the identification code Dcode is input from the control unit 133, the output unit 135 converts the input identification code Dcode into output data Dout in a predetermined data format and outputs it to the external device 300. The external device 300 performs predetermined information processing using the output data Dout input from the image processing device 100. In this manner, image processing in the image processing device 100 is performed.
[0076] In this modification, when the matching rate does not satisfy the determination condition 132d, a process is performed to indicate that the identification code Dcode is not correctly read data, and when the matching rate satisfies the determination condition 132d, the identification code Dcode is output to the output unit 135. This makes it possible to prevent the external device 300 from performing a process based on an incorrect identification code Dcode.
[0077] [Modification E] In the above embodiment and its modifications, the image processing unit 130 may have an image processing unit 140 instead of the image processing unit 130, as shown in Fig. 17. The image processing unit 140 may be the same as the image processing unit 130 shown in Fig. 2, except that the communication unit 136 is omitted and a storage unit 220 is further provided, as shown in Fig. 18.
[0078] For example, as shown in FIG. 19 , the image processing unit 140 may be the image processing unit 130 shown in FIG. 5 , except that the communication unit 136 is omitted and storage units 220 and 250 are further provided. For example, as shown in FIG. 20 , the image processing unit 140 may be the image processing unit 130 shown in FIG. 9 , except that the communication unit 136 is omitted and storage unit 220 is further provided. For example, as shown in FIG. 21 , the image processing unit 140 may be the image processing unit 130 shown in FIG. 13 , except that the communication unit 136 is omitted and storage unit 220 is further provided. For example, as shown in FIG. 22 , the image processing unit 140 may be the image processing unit 130 shown in FIG. 15 , except that the communication unit 136 is omitted and storage unit 220 is further provided.
[0079] In this modification, the storage unit 200 (and further the storage unit 250) that was provided in the learning device 200 is provided in the image processing device 100. Even in this case, it is possible to obtain the same effects as those of the above embodiment and its modifications.
[0080] It should be noted that the effects described in this specification are merely examples. The effects of the present disclosure are not limited to the effects described in this specification. The present disclosure may have effects other than those described in this specification. Furthermore, although examples are described separately in this specification, it goes without saying that these examples may be combined.
[0081] Furthermore, for example, the present disclosure may be configured as follows: (1) An image processing device comprising: a receiving unit, a first learning model, and a decoding unit, wherein the receiving unit is capable of receiving a code image obtained by imaging a code on a workpiece, wherein the first learning model is a machine-learned model using as training data a first code image obtained by imaging and a SemanticSegmentation image corresponding to the first code image that has been subjected to two-class classification, and when the code image or a processed image obtained by performing a predetermined process on the code image is input, the image processing device is capable of outputting a binary image corresponding to the code image or the processed image, and the decoding unit is capable of generating an identification code by decoding the binary image. (2) The image processing device described in (1), further comprising: a second learning model, wherein the second learning model is a machine-learned model using as training data a second code image obtained by imaging and a defect image obtained by adding a defect to the second code image, and when the code image is input, the image in which the defect included in the code image has been repaired is output as the processed image.(3) The image processing device according to (1) or (2), further comprising an imaging support unit and a learning unit, wherein the imaging support unit is capable of repeatedly outputting a control signal, when the decoding unit is unable to generate the identification code, instructing an imaging device capable of generating the code image to change imaging conditions, while changing the imaging conditions until the decoding unit is able to generate the identification code; the learning unit has a master learning model which is backup data of the first learning model, and is capable of replacing the first learning model with a new learning model obtained by performing machine learning on the master learning model; and the learning unit is capable of generating the new learning model by performing machine learning on the master learning model, with the code image obtained by the imaging device as a result of the imaging support unit changing the imaging conditions as the first code image, and the binary image output from the first learning model as a result of the imaging support unit changing the imaging conditions as the SemanticSegmentation image. (4) The image processing device according to any one of (1) to (3), further comprising a determination unit, wherein the determination unit is capable of outputting the identification code to an external device when the identification code generated by the decoding unit satisfies a predetermined condition, and performing processing to indicate that the identification code is not correctly read data when the identification code generated by the decoding unit does not satisfy the predetermined condition. (5) The image processing device according to any one of (1) to (3), further comprising a determination unit, wherein the determination unit is capable of outputting the identification code to an external device when a matching rate between the binary image obtained from the first learning model and a binary image for determination obtained by encoding the identification code generated by the decoding unit satisfies a predetermined condition, and performing processing to indicate that the identification code is not correctly read data when the matching rate does not satisfy the predetermined condition.(6) An image processing method including: acquiring a code image obtained by imaging a code on a workpiece; inputting the code image or a processed image obtained by performing a predetermined process on the code image into a first learning model that has been subjected to machine learning using, as learning data, a first code image obtained by imaging and a SemanticSegmentation image that has been subjected to two-class classification corresponding to the first code image, thereby acquiring a binary image corresponding to the code image or the processed image from the first learning model; and generating an identification code by decoding the binary image. (7) An image processing program that includes a first learning model that is machine-learned using as learning data a first code image obtained by imaging and a SemanticSegmentation image that has been classified into two classes and corresponds to the first code image, and that is capable of outputting a binary image corresponding to an input image when the input image is input, and that can cause a computer to execute the following operations: acquiring a code image obtained by imaging a code on a workpiece; and inputting the code image or a processed image that has been subjected to a predetermined process on the code image into the first learning model as the input image, thereby acquiring an image corresponding to the code image or the processed image from the first learning model as the binary image.
[0082] In an image processing device according to a first aspect of the present disclosure, an image processing method according to a second aspect of the present disclosure, and a recording medium according to a third aspect of the present disclosure, a feature image is generated by encoding a region image including a subject extracted from a captured image, and a mask image is generated by combining the captured image and the feature image. Here, the feature image included in the mask image does not serve as information identifying the subject. Therefore, information identifying the subject cannot be obtained from the mask image, and therefore, even if the mask image is provided to an external party, information identifying the subject can be prevented from leaking to the external party. Furthermore, M-dimensional feature data obtained by decoding the feature image using a known decoder contains features unique to the subject. Therefore, the subject can be identified by analyzing the M-dimensional feature data obtained from the feature image. Therefore, a process can be performed to make it difficult to identify information identifying the subject without impairing the subject recognition function.
[0083] This application claims priority based on Japanese Patent Application No. 2023-089836, filed on May 31, 2023, in the Japan Patent Office, the entire contents of which are incorporated herein by reference.
[0084] Those skilled in the art will recognize that various modifications, combinations, subcombinations, and variations may occur depending on design requirements and other factors, and are intended to be within the scope of the appended claims and their equivalents.
Claims
1. An image processing device comprising a reception unit, a first learning model, and a decoding unit, wherein the reception unit is capable of receiving a code image obtained by imaging a code on a workpiece, the first learning model is a model that has been subjected to machine learning using as learning data a first code image obtained by imaging and a SemanticSegmentation image that has been subjected to two-class classification corresponding to the first code image, and when the code image or a processed image in which a predetermined process has been performed on the code image is input, the image processing device is capable of outputting a binary image corresponding to the code image or the processed image, and the decoding unit is capable of generating an identification code by decoding the binary image.
2. The image processing device according to claim 1, further comprising a second learning model, the second learning model being a model that has been machine-learned using as learning data a second code image obtained by imaging and a defective image in which a defect has been added to the second code image, and when the code image is input, it is capable of outputting an image in which the defect contained in the code image has been repaired as the processed image.
3. An image processing device as described in claim 1, further comprising an imaging support unit and a learning unit, wherein when the decoding unit is unable to generate the identification code, the imaging support unit is capable of repeatedly outputting a control signal instructing an imaging device capable of generating the code image to change the imaging conditions while changing the imaging conditions until the decoding unit can generate the identification code, and the learning unit has a master learning model which is backup data of the first learning model and is capable of replacing the first learning model with a new learning model obtained by performing machine learning on the master learning model, and the learning unit is capable of generating the new learning model by performing machine learning on the master learning model, with the code image obtained by the imaging device as a result of the change in the imaging conditions by the imaging support unit being the first code image, and the binary image output from the first learning model as a result of the change in the imaging conditions by the imaging support unit being the SemanticSegmentation image.
4. An image processing device as described in claim 1, further comprising a judgment unit which is capable of outputting the identification code generated by the decoding unit to an external device when the identification code satisfies a predetermined condition, and performing processing to indicate that the identification code is not correctly read data when the identification code generated by the decoding unit does not satisfy the predetermined condition.
5. An image processing device as described in claim 1, further comprising a judgment unit, which is capable of outputting the identification code to an external device when a matching rate between the binary image obtained from the first learning model and a binary image for judgment obtained by encoding the identification code generated by the decoding unit satisfies a predetermined condition, and performing processing to indicate that the identification code is not correctly read data when the matching rate does not satisfy the predetermined condition.
6. An image processing method comprising: acquiring a code image obtained by imaging a code on a workpiece; inputting the code image or a processed image obtained by performing a predetermined process on the code image into a first learning model that has been subjected to machine learning using as learning data a first code image obtained by imaging and a SemanticSegmentation image that has been subjected to a two-class classification corresponding to the first code image, thereby acquiring a binary image corresponding to the code image or the processed image from the first learning model; and generating an identification code by decoding the binary image.
7. An image processing program capable of causing a computer to execute the following operations: obtaining a code image obtained by imaging a code on a workpiece; and inputting, as the input image, a processed image obtained by performing a predetermined process on the code image into the first learning model, the first learning model being a machine-learned model using as learning data a first code image obtained by imaging and a SemanticSegmentation image corresponding to the first code image that has been subjected to a two-class classification. The image processing program is capable of causing a computer to execute the following operations: obtaining a code image obtained by imaging a code on a workpiece; and inputting, as the input image, the code image or a processed image obtained by performing a predetermined process on the code image into the first learning model, thereby obtaining, as the binary image, an image corresponding to the code image or the processed image from the first learning model.
Citation Information
Patent Citations
Individual identification method of product
JP2021140295A
Optical information reader
JP2022111038A
Information code reader, game device, information code reading method, computer program, and method of generating learning model
JP2023006539A
Information reading device
WO2022211064A1
