Information code reading device, game machine, information code reading method, computer program, and learning model generation method

Abstract

To provide an information code reader, a game device, an information code reading method, a computer program, and a method of generating a learning model.SOLUTION: An information code reader includes: an acquisition unit which acquires a captured image of an information code; a conversion unit which converts the captured image acquired by the acquisition unit into a reproduction image of a correct answer image, using a learning model trained to output a reproduction image that reproduces a correct answer image of an information code when the captured image of the information code is input; and a read unit which reads information indicated by the information code from the reproduction image converted by the conversion unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information code reading device, a game machine, an information code reading method, a computer program, and a learning model generation method. [Background technology]

[0002] Information code reading devices include one-dimensional code readers that read one-dimensional codes such as barcodes, and two-dimensional code readers that read two-dimensional codes such as QR Code (registered trademark). These code readers include an optical sensor (such as an image sensor) that acquires optical information from information codes such as one-dimensional codes and two-dimensional codes, converts the electrical signal (analog signal) output from the optical sensor into a digital signal, and decodes the digital signal to read the information code.

[0003] In order to improve the accuracy of recognizing information codes, Patent Document 1 discloses a pattern recognition device that analyzes a captured image, adjusts image quality such as resolution and contrast, and performs pattern recognition based on the corrected image after the image quality adjustment.

[0004] Cited Document 2 discloses an image recognition device that acquires a grayscale image of an information code, enhances white and black by setting thresholds for the shades of the acquired grayscale image, and recognizes the information code based on the image with improved contrast. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-277315 [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-96095 Summary of the Invention [Problem to be solved by the invention]

[0006] However, if factors that hinder the reading of the information code occur, such as printing misalignment or blurring when printing the information code on a medium, deterioration of the information code image due to aging, or exposure to external light when capturing an image, the information code may not be readable even if the image quality adjustments described above are performed.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide an information code reading device, a game machine, an information code reading method, a computer program, and a method for generating a learning model that can read an information code based on an image of the acquired information code even if the image contains an obstructing factor. [Means for solving the problem]

[0008] An information code reading device according to one embodiment of the present invention includes an acquisition unit that acquires an image of an information code, a conversion unit that converts the image acquired by the acquisition unit into a reproduction image of a correct image using a learning model that has been trained to output a reproduction image that reproduces a correct image of the information code when the image of the information code is input, and a reading unit that reads information indicated by the information code from the reproduction image converted by the conversion unit. [Effects of the Invention]

[0009] According to the present application, even if an obstruction factor is included in the captured image of the acquired information code, the information code can be read based on the captured image. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is an explanatory diagram illustrating an outline of an information code reading process according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of an information code reader; [Figure 3]FIG. 10 is an explanatory diagram illustrating a learning model used in the conversion process from a captured image to a reproduced image. [Figure 4] 3 is a flowchart illustrating a processing procedure according to the first embodiment. [Figure 5] 10 is a flowchart illustrating a processing procedure according to the second embodiment. [Figure 6] FIG. 10 is a schematic diagram showing an example of a processed image. [Figure 7] 10 is a flowchart illustrating a procedure for generating a learning model. [Figure 8] 10 is a flowchart illustrating a procedure for additional learning in the operation phase. [Figure 9] FIG. 11 is a block diagram showing the configuration of a game machine according to a fifth embodiment. [Figure 10] FIG. 10 is a conceptual diagram illustrating an attribute table. [Figure 11] 10 is a flowchart illustrating an example of a processing procedure in the game machine. [Figure 12] 10 is a flowchart showing another example of the processing procedure in the game machine. DETAILED DESCRIPTION OF THE INVENTION

[0011] The present invention will now be described in detail with reference to the drawings showing embodiments thereof. (Embodiment 1) FIG. 1 is an explanatory diagram outlining an information code reading process in the first embodiment. The information code reader 10 is a device for acquiring a captured image of an information code and reading information indicated by the information code from the acquired captured image. The information code may be a one-dimensional code such as a barcode, or a two-dimensional code such as a QR code (registered trademark). Hereinafter, codes recorded as images, such as barcodes and QR codes, are collectively referred to as information codes. The captured image of the information code is a digital image obtained by capturing, with the imaging device 20, an image of an information code printed on a medium such as a predetermined recording paper or card. The captured image of the information code may also be a digital image obtained by capturing, with the imaging device 20, an image of an information code displayed on the screen of a smartphone or tablet terminal.

[0012] Here, if the captured image contains factors that inhibit reading of the information code (hereinafter referred to as reading-inhibiting factors), the reading device may not be able to read the contents of the information code. For example, if printing misalignment or blurring occurs when printing the information code on a medium, or if the image has deteriorated over time, the reading device may not be able to properly read the contents of the information code. Furthermore, if noise is superimposed during imaging, or if the contrast is reduced overall or partially due to the influence of external light during imaging, the reading device may not be able to properly read the contents of the information code.

[0013] When an image of an information code is input, the information code reading device 10 of this embodiment converts the input image into a reproduction image using a learning model MD that has been trained to output a reproduction image that reproduces the correct image of the information code, and reads the information indicated by the information code from the converted reproduction image.

[0014] 2 is a block diagram showing the configuration of the information code reader 10. The information code reader 10 is a dedicated or general-purpose computer, and includes a control unit 11, a storage unit 12, an input unit 13, and an output unit 14, for example.

[0015] The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The ROM included in the control unit 11 stores control programs and the like for controlling the operation of each of the above hardware components. The CPU in the control unit 11 executes the control programs stored in the ROM and various programs stored in the storage unit 12, and controls the operation of each of the above hardware components, thereby causing the entire device to function as the information code reading device of the present invention. The RAM included in the control unit 11 stores data that is temporarily used while the various programs are being executed.

[0016] Note that the control unit 11 is not limited to the above configuration, and may be one or more processing circuits or arithmetic circuits including a single-core CPU, a multi-core CPU, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), a microcomputer, a volatile or non-volatile memory, etc. Furthermore, the control unit 11 may also have functions such as a clock that outputs date and time information, a timer that measures the elapsed time from when an instruction to start measurement is given until when an instruction to end measurement is given, and a counter that counts numbers.

[0017] The storage unit 12 includes storage devices such as a hard disk drive (HDD) and a solid state drive (SSD). The storage unit 12 stores various programs to be executed by the control unit 11. The programs stored in the storage unit 12 include a conversion program PG1 that converts an input captured image into a reproduction image using a learning model MD that has been trained to output a reproduction image that reproduces a correct image of the information code when the captured image of the information code is input, and a reading program PG2 that reads information indicated by the information code from the converted reproduction image. The conversion program PG1 and the reading program PG2 may be independent computer programs or may be an integrated computer program. The conversion program PG1 and the reading program PG2 may also be plug-ins or libraries that can be used by an existing computer.

[0018] The computer program stored in the storage unit 12 is provided by a storage medium RM on which the computer program is readably recorded. The storage medium RM is, for example, a portable memory such as an SD (Secure Digital) card, a micro SD card, or a CompactFlash (registered trademark). The control unit 11 reads the program from the storage medium RM using a reading device (not shown) and stores the read program in the storage unit 12. The computer program stored in the storage unit 12 may also be provided via communication. In this case, the control unit 11 simply obtains the program provided via communication and stores the obtained program in the storage unit 12. For this reason, the information code reading device 10 may be provided with a communication unit (communication interface) for communicating with external devices.

[0019] The storage unit 12 further includes a learning model MD that is trained to output a reproduction image that reproduces a correct image of the information code when a captured image of the information code is input. The storage unit 12 stores information defining the learning model MD, such as configuration information of layers included in the learning model MD, information on nodes (neurons) included in each layer, and information on weighting and bias between nodes. The specific configuration of the learning model MD will be described in detail later.

[0020] The input unit 13 includes a connection interface for connecting an external device capable of outputting a captured image of an information code. The external device connected to the input unit 13 is, for example, an imaging device 20 such as a digital camera or video camera that captures an information code printed on a medium or an information code displayed on a screen. The control unit 11 acquires, from the input unit 13, the captured image of the information code captured by the imaging device 20. Alternatively, the input unit 13 may be connected to an external computer such as a smartphone or a personal computer. The control unit 11 may acquire, from the input unit 13, the captured image of the information code imported into the external computer.

[0021] The output unit 14 has a connection interface to which a processing device 30 is connected, which executes appropriate processing based on the information read from the information code. The processing device 30 includes, for example, a smartphone, a tablet terminal, a personal computer, a server device, a portable game machine, an arcade game machine, etc. The processing device 30 may perform electronic payment processing based on the information output from the output unit 14. The processing device 30 may also perform processing to determine attribute information of characters appearing in the game based on the information output from the output unit 14.

[0022] The information code reading device 10 does not have to be a single computer, but may be a computer system configured with multiple computers and peripheral devices. The information code reading device 10 may also be a virtual machine virtually constructed by software. Furthermore, the information code reading device 10 may be a device incorporated into other devices such as smartphones, tablet terminals, personal computers, server devices, portable game consoles, and arcade game consoles.

[0023] The process executed by the information code reader 10 will be described below. When the information code reader 10 acquires a captured image of an information code, it converts the acquired captured image into a reproduced image of the correct image using the learning model MD.

[0024] FIG. 3 is an explanatory diagram illustrating a learning model MD used in the conversion process from a captured image to a reproduced image. The learning model MD is constructed, for example, by a GAN (Generative Adversarial Network). The GAN includes a generator MD1 and a discriminator MD2. The generator MD1 is a neural network that generates a reproduced image that reproduces a correct image from a captured image of an information code. Here, the reproduced image represents an image obtained by estimating the original image (correct image) of the information code from the captured image. In other words, the generator MD1 removes reading-impeding factors such as noise, degradation, poor contrast, and geometric distortion from the captured image, and generates an image (reproduced image) that is close to the correct image of the information code. As learning progresses, the generator MD1 is configured to generate a reproduced image that is closer to the correct image.

[0025] The classifier MD2 is a neural network that outputs a predicted result of whether an input image is true or false. The classifier MD2 is trained to output a predicted result of "true" when a correct answer image prepared as training data is input, and to output a predicted result of "false" when a reproduced image generated by the generator MD1 is input.

[0026] In the GAN that constructs the learning model MD, a loss function based on the accuracy of the prediction results is set for both the generator MD1 and the classifier MD2. During the learning process, the parameters of each network (weights and biases between nodes) are sequentially updated so that the loss function of the generator MD1 is ultimately maximized and the loss function of the classifier MD2 is minimized. In other words, in the GAN, learning progresses by competing with each other so that the generator MD1 generates a fake (reproduced image) that is closer to the correct image, and the classifier MD2 correctly classifies the input image.

[0027] Pix2pix can be used as an example of a GAN that generates a reproduced image from a captured image of an information code. Regarding pix2pix, the one disclosed in "Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A. Efros: Image-to-Image Translation with Conditional Adversarial Networks, arXiv preprint arXiv: 1611.07004 (2016)" can be used. Instead of pix2pix, any appropriate image-generating neural network, such as PGGAN (Progressive Growing of GANs) or CycleGAN, can also be used. Furthermore, instead of being limited to image-generating neural networks, any neural network capable of image segmentation, such as SegNet, FCN (Fully Convolutional Network), U-Net (U-Shaped Network), or PSPNet (Pyramid Scene Parsing Network), can also be used. Furthermore, it may be constructed using a neural network for object detection such as YOLO (You Only Look Once) or SSD (Single Shot Multi-Box Detector).

[0028] 3, the learning model MD is described as being composed of a generator MD1 and a classifier MD2. However, in the operation phase after learning is completed, since it is only necessary to generate a reproduced image from a captured image, only inference by the generator MD1 may be performed. Furthermore, the learning model MD stored in the storage unit 12 of the information code reading device 10 may include a neural network corresponding to the trained generator MD1, but may not include a neural network corresponding to the classifier MD2. In this case, it is sufficient that definition information of the neural network corresponding to the generator MD1 is stored in the storage unit 12.

[0029] The control unit 11 of the information code reading device 10 inputs a captured image of the information code into the learning model MD and executes calculations using the learning model MD to obtain a reproduced image output from the generator MD1. The control unit 11 reads information indicated by the information code from the obtained reproduced image. Existing technology is used to read the information code. For example, when reading a two-dimensional code consisting of white and black cells, the control unit 11 recognizes the horizontal and vertical directions of the code from the position detection pattern included in the two-dimensional code, recognizes the white and black cells within the cell area, rearranges the recognized white cells into a data string of 0s and black cells into a string of 1s, and performs a decoding process to convert the recognized white cells into character information. The control unit 11 can read information codes by using a reading method determined according to the type of code, not limited to the above-mentioned two-dimensional code. The control unit 11 outputs information indicated by the read information code (character information) from the output unit 14 to an external processing device 30.

[0030] The procedure of the process executed by the information code reader 10 will be described below. FIG. 4 is a flowchart illustrating a processing procedure in the first embodiment. The control unit 11 of the information code reading device 10 acquires a captured image of an information code through the input unit 13 (step S101). The captured image of an information code is a digital image obtained by capturing an image of an information code printed on a medium or an information code displayed on a screen. The control unit 11 may acquire the captured image of an information code directly from the imaging device 20, or may acquire, from the external device, a captured image of an information code that has been imported into the external device. The acquired captured image is temporarily stored in the memory unit 12.

[0031] The control unit 11 inputs the acquired captured image into the learning model MD and converts the captured image into a reproduced image by performing calculations using the learning model MD (step S102). That is, by inputting a captured image of an information code into the generator MD1, a reproduced image that reproduces the correct image of the information code is obtained from the generator MD1. Note that, when inputting the captured image into the learning model MD, the control unit 11 may also perform a region extraction process to extract a rectangular region surrounding the information code, and input only the image of the extracted region into the learning model MD.

[0032] The control unit 11 reads the information code from the reproduced image obtained by converting the captured image (step S103). The control unit 11 can read the information (character information) indicated by the information code by using existing technology. For example, the control unit 11 can recognize the shading that changes in the horizontal direction (and / or vertical direction) of the information code as a data string, and decode the recognized data string according to a predetermined rule, thereby reading the information (character information) indicated by the information code.

[0033] The control unit 11 outputs the information indicated by the information code read from the reproduced image to the processing device 30 from the output unit 14 (step S104). The information output from the output unit 14 is used by the processing device 30 at the output destination. For example, if the processing device 30 is a general-purpose computer such as a smartphone or tablet terminal, the processing device 30 can execute a process of displaying the information indicated by the information code on a screen, a process of transitioning to a homepage specified by the information indicated by the information code, an electronic payment process based on the information indicated by the information code, etc. If the processing device 30 is a game machine such as a portable game machine or an arcade game machine, it may execute a process of determining attribute information of a character to be featured in a game based on the information indicated by the information code, and having a character having the attribute information appear in the game.

[0034] As described above, when an image of an information code is input, the information code reading device 10 of embodiment 1 converts the image into a reproduction image using a learning model MD that has been trained to output a reproduction image that reproduces the correct image of the information code, and reads the information indicated by the information code from the converted reproduction image.Therefore, even if the image contains factors that hinder reading, the information indicated by the information code can be read well.

[0035] In this embodiment, the information code reading device 10 is configured to include the learning model MD, but the learning model MD may be stored in an external server. In this case, the information code reading device 10 may be configured to transmit a captured image of the information code to the external server and obtain information on the read information code from the external server.

[0036] (Embodiment 2) In the second embodiment, a configuration will be described in which an information code included in a captured image is read, and if the reading fails, the captured image is converted into a reproduced image. The internal configuration of the information code reader 10 and the configuration of the learning model MD are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0037] 5 is a flowchart illustrating a processing procedure in the second embodiment. The control unit 11 of the information code reading device 10 acquires a captured image of an information code through the input unit 13 (step S201), and reads the information code from the acquired captured image (step S202). The control unit 11 can read information (text information) indicated by the information code by using existing technology. As a preprocessing before reading the information code, the control unit 11 may execute an area extraction process for extracting a rectangular area surrounding the information code from the captured image.

[0038] The control unit 11 determines whether or not the information code has been successfully read (step S203). If predetermined character information is obtained from the information code, the control unit 11 can determine that the information code has been successfully read. If it is determined that the information code has been successfully read (S203: YES), the control unit 11 executes the processes from step S206 onwards, which will be described later.

[0039] If it is determined that reading of the information code has failed (S203: NO), the control unit 11 inputs the captured image acquired in step S201 into the learning model MD and converts the captured image into a reproduced image by performing calculations using the learning model MD (step S204). The conversion method is the same as in embodiment 1. When inputting the captured image into the learning model MD, the control unit 11 may also perform area extraction processing to extract a rectangular area surrounding the information code, and input an image of the extracted area into the learning model MD.

[0040] The control unit 11 reads the information code from the reproduced image obtained by converting the captured image (step S205). The reading method is the same as in embodiment 1. By reading the information code, the control unit 11 obtains the information (text information) indicated by the information code.

[0041] The control unit 11 outputs the information indicated by the information code read from the reproduction image from the output unit 14 to the processing device 30 (step S206). The information output from the output unit 14 is used by the processing device 30 as the output destination, as in the first embodiment.

[0042] As described above, in the second embodiment, the captured image is converted into a reproduced image only when reading of the information code fails, so that the processing load on the information code reader 10 can be reduced.

[0043] (Embodiment 3) In the third embodiment, a method for generating a learning model MD will be described.

[0044] The information code reading device 10 generates a learning model MD by using a data set including multiple pairs of captured images of an information code and correct images of the information code as training data and performing learning using a predetermined algorithm such as pix2pix.

[0045] However, it is difficult to prepare captured images that take into account various reading-impeding factors. Therefore, in this embodiment, a learning model MD is generated by preparing a large number of processed images in which various reading-impeding factors are added to captured images or correct images, and using a data set that further includes the processed images and correct images as training data and performing learning using a predetermined algorithm such as pix2pix.

[0046] FIG. 6 is a schematic diagram showing an example of a processed image. The original image shown in FIG. 6 is a correct image of the information code. A captured image of the information code may be used instead of the correct image. That is, the original image to be processed may be either a correct image or a captured image. In the example of FIG. 6, the processed images shown are a contrast-changed image, a shift-moved image, a geometrically transformed image, and a noise-added image. The contrast-changed image is an image in which the contrast of the information code is reduced and the difference in shading between black cells and white cells is reduced. The shift-moved image is an image in which the information code portion is shifted horizontally (or vertically) and part of the information code is missing. The geometrically transformed image is an image in which a geometric transformation (trapezoidal distortion in the example of FIG. 6) is applied to the information code portion. The noise-added image is an image in which noise represented by black dots is superimposed on the original image.

[0047] These processed images are generated by using an image processing application on a computer. The processed images are not limited to those shown in FIG. 6. For example, an image with modified resolution or density may be prepared as the processed image, taking into account the resolution and print density of the printing device that prints the information code. Furthermore, an image with partially reduced contrast may be prepared as the processed image, taking into account the lighting of the installation environment. Furthermore, an image with superimposed noise may be prepared as the processed image, taking into account deterioration of the imaging device 20 (e.g., dirt on the lens or reflector). Furthermore, an image with superimposed noise or blurring may be prepared as the processed image, taking into account deterioration of the medium on which the information code is printed. Furthermore, an image that has undergone geometric transformation (projective transformation) including translation, rotation, and trapezoidal transformation may be prepared as the processed image, taking into account deformation of the medium on which the information code is printed.

[0048] The captured image, processed image, and correct image of the prepared information code are stored in the memory unit 12. The control unit 11 uses these images as training data to train the generator MD1 and the classifier MD2, thereby generating a learning model MD.

[0049] 7 is a flowchart illustrating the procedure for generating the learning model MD. The control unit 11 reads one of the captured images or processed images included in the training data from the storage unit 12, and inputs the read image to the generator MD1 (step S301). The control unit 11 converts the input image into a reproduced image by executing a calculation in the generator MD1 (step S302). Note that, at the initial stage of starting learning, it is assumed that initial values ​​are set for the parameters that characterize the generator MD1 (weights and biases between nodes of the neural network).

[0050] The control unit 11 inputs the reproduced image converted in step S302 or the correct image read from the storage unit 12 to the classifier MD2 (step S303). It is assumed that the control unit 11 knows whether the input image is a correct image (true) or a reproduced image (false). The control unit 11 executes a calculation in the classifier MD2 to obtain a classification result (step S304). It is assumed that initial values ​​are set for the parameters characterizing the classifier MD2 (weights and biases between nodes of the neural network) at the initial stage of starting learning.

[0051] The classification result by classifier MD2 indicates the predicted result of the authenticity of the image input in step S303. If the input image is a correct image and the predicted result is "true", or if the input image is a reproduced image and the predicted result is "false", the predicted result is determined to be correct. On the other hand, if the input image is a correct image and the predicted result is "false", or if the input image is a reproduced image and the predicted result is "true", the predicted result is determined to be incorrect.

[0052] In GAN, a loss function based on the accuracy of the prediction result is set for both the generator MD1 and the classifier MD2. The control unit 11 calculates the loss functions for both the generator MD1 and the classifier MD2 (step S305) and determines whether learning is complete (step S306). That is, the control unit 11 may determine that learning is complete when the loss function for the generator MD1 is maximized and the loss function for the classifier MD2 is minimized. Note that to avoid the problem of overfitting, techniques such as cross-validation and early termination may be adopted to terminate learning at an appropriate time.

[0053] If it is determined that the learning is not complete (S306: NO), the control unit 11 changes the parameters (weights and biases between nodes of the neural network) of the generator MD1 and the discriminator MD2 (step S307), and returns the process to step S301.

[0054] If it is determined that the learning is completed (S306: YES), the control unit 11 ends the processing according to this flowchart.

[0055] As described above, in embodiment 3, by performing learning using the captured image of the information code, the processed image, and the correct image as training data, it is possible to generate a learning model MD (generator MD1) that is configured to output a reproduced image that reproduces the correct image when the captured image of the information code is input.

[0056] In the third embodiment, the learning model MD is generated inside the information code reading device 10, but the learning model MD may be generated in an external server. The information code reading device 10 acquires the learning model MD generated in the external server by means of communication or the like, and installs the acquired learning model MD in the storage unit 12.

[0057] Furthermore, the flowchart shown in Figure 7 explains the procedure for preparing a captured image (and processed image) of an information code and a correct image, and generating a learning model MD from scratch. However, if you want to deal with additional reading-impeding factors, you can do so by preparing new training data that includes the additional reading-impeding factors and performing transfer learning on the already-learned learning model MD.

[0058] (Fourth embodiment) In the fourth embodiment, additional learning in the operation phase will be described.

[0059] Since captured images of the information code can be obtained even in the operation phase, the information code reading device 10 may store the captured images obtained in the operation phase and use the stored captured images as training data to perform additional learning of the learning model MD. Also, the information code reading device 10 may present information read from the reproduced image to the user and ask the user to confirm whether the read information is correct or not.

[0060] 8 is a flowchart illustrating the procedure for additional learning in the operation phase. The control unit 11 of the information code reading device 10 acquires a captured image of the information code, converts it into a reproduced image by performing calculations using the learning model MD, and reads the information code from the reproduced image in the same procedure as in the first embodiment (steps S401 to S403).

[0061] The control unit 11 outputs the information obtained by reading the information code from the output unit 14 and displays it on the display screen of the processing device 30 (step S404). If the user determines that the displayed information is incorrect, the control unit 11 accepts a correction of the information through the processing device 30 (step S405). If the information to be displayed on the display screen of the processing device 30 is a character in a game, multiple characters may be presented as options, and the user may select the correct character.

[0062] The control unit 11 associates the captured image acquired in step S401 with the correct image of the information code indicating the information corrected in step S405 and stores them in the storage unit 12 (step S406). The correct image of the information code indicating the corrected information can be obtained by encoding the corrected information using an appropriate method.

[0063] The control unit 11 performs additional learning of the learning model MD using the captured image and the correct image stored in step S406 as training data (step S407). Note that the control unit 11 does not need to perform additional learning every time it acquires an additional captured image and a correct image, and it only needs to perform additional learning when a sufficient number of training data is obtained.

[0064] As described above, in the fourth embodiment, the captured images obtained during operation are accumulated and the learning model MD is additionally learned, thereby improving accuracy.

[0065] (Embodiment 5) In the fifth embodiment, an application example in an arcade-type game machine (hereinafter simply referred to as a game machine) will be described.

[0066] 9 is a block diagram showing the configuration of a game machine 100 according to embodiment 5. The game machine 100 according to embodiment 5 is a game machine that reads, for example, an information code recorded on a game card (medium), causes a character corresponding to the read information to appear in the game, and progresses a predetermined scenario. The game machine 100 includes a control unit 101, a storage unit 102, an operation unit 103, a display unit 104, a deposit acceptance unit 105, a card issuance unit 106, an imaging unit 107, a communication unit 108, an information code reading device 10, etc.

[0067] The control unit 101 includes, for example, a CPU, a ROM, and a RAM. The ROM included in the control unit 101 stores a control program that controls the operation of each hardware unit included in the game console 100, data necessary for control, etc. The CPU in the control unit 101 executes the control program stored in the ROM and controls the operation of each hardware unit. The RAM included in the control unit 101 temporarily stores various data generated during execution of control.

[0068] The memory unit 102 is configured with a storage device such as a flash memory. The memory unit 102 stores various computer programs and data used in the game console 100. The computer programs stored in the memory unit 102 include a program for causing the control unit 101 to progress the game scenario. The data stored in the memory unit 102 includes attribute information of characters that appear in the game. The character attribute information is managed by an attribute table conceptually shown in FIG. 10. The character attribute information includes information such as the character's name, design, rarity, strength, and technique. In this embodiment, an information code is assigned to each type of character, and the attribute information of each character is stored in the attribute table in association with the information code.

[0069] The operation unit 103 is equipped with various buttons, and outputs information corresponding to pressed buttons to the control unit 101. The buttons equipped on the operation unit 103 include, for example, a start button for starting the game, a selection button for accepting selection operations by the player, and the like. The buttons equipped on the operation unit 103 are designed as appropriate depending on the model, game scenario, and the like. The operation unit 103 may also be equipped with a touch panel. If the operation unit 103 is equipped with a touch panel, the operation unit 103 outputs information input from the touch panel to the control unit 101.

[0070] The display unit 104 is equipped with a display device such as an LCD monitor or an organic EL (Electro-Luminescence) display, and displays a selection screen for accepting selection operations by the player and a game screen showing the progress of the game in response to instructions from the control unit 101.

[0071] The deposit acceptance unit 105 includes an insertion slot into which currency is inserted, an identification device that identifies the inserted currency, a storage unit that stores the inserted currency, and an outlet that dispenses change. The deposit acceptance unit 105 identifies currency inserted into the insertion slot using the identification device, stores the identified currency in the storage unit, and outputs information on the identified amount (amount information) to the control unit 11. The deposit acceptance unit 105 dispenses currency (change) in an amount obtained by deducting the game play fee from the identified amount, to the outlet. The deposit acceptance unit 105 may be configured to accept only fixed-denomination currency so that change is not required. In this case, the deposit acceptance unit 105 does not need to be provided with an outlet. The deposit acceptance unit 105 may also be configured to accept deposits by credit card, prepaid card, postpay card, or electronic payment.

[0072] The card issuing unit 106 includes a storage unit that stores card mounts, a printing unit that prints information codes on card mounts removed from the storage unit, and an outlet through which game cards completed by printing information codes on card mounts are ejected. Existing printing methods, such as thermal printing, inkjet printing, and laser printer printing, are used to print the information codes. Each time the deposit accepting unit 105 accepts a predetermined play fee, or each time the deposit accepting unit 105 accepts a predetermined play fee and the game ends (or a predetermined clearing condition is met), the card issuing unit 106 prints the information code on the card mount to create a game card, and ejects the created game card from the outlet. The information code to be printed on the game mount may be selected randomly or based on the content of the game.

[0073] The imaging unit 107 includes a platform on which a game card is placed, and an imaging device that captures an image of the game card placed on the platform. The imaging device includes an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge-Coupled Device), and generates a captured image by capturing an image of the information code printed on the game card. Note that the imaging range of the imaging device does not need to be the entire game card, and may be limited to the range of the information code.

[0074] The communication unit 108 includes a communication interface for communicating with an external device. The communication interface included in the communication unit 108 is, for example, a communication interface such as WiFi (registered trademark), LAN (Local Area Network), Bluetooth (registered trademark), WiFi (registered trademark), ZigBee (registered trademark), 3G, 4G, 5G, or LTE (Long Term Evolution). The communication unit 108 transmits various data to be notified to the external device and receives various data transmitted from the external device. An external device communicatively connected to the communication unit 108 is, for example, a management server 200 that manages multiple game consoles 100 installed in each arcade. Alternatively, an external device communicatively connected to the communication unit 108 may be an external server that generates a learning model MD.

[0075] The information code reading device 10 is similar to those described in the first to fourth embodiments, and includes a control unit 11, a storage unit 12, an input unit 13, and an output unit 14. A captured image of an information code captured by an imaging unit 107 is input to the input unit 13 of the information code reading device 10. The control unit 11 inputs the captured image input by the input unit 13 to a learning model MD and performs calculations using the learning model MD to convert the captured image into a reproduced image. The control unit 11 obtains information indicated by the information code by decoding the reproduced image converted by the learning model MD.

[0076] The procedure for reading an information code by the information code reader 10 is the same as in the embodiment described above. That is, when a captured image of an information code is input, the information code reader 10 converts the captured image into a reproduced image each time, and reads information indicated by the information code from the reproduced image after the conversion. Alternatively, when a captured image of an information code is input, the information code reader 10 may read the information code from the captured image, and only if the reading fails, convert it into a reproduced image, and read information indicating the information code from the reproduced image after the conversion.

[0077] Since the game machine 100 is installed in various stores, the lighting conditions when the information code is captured depend on the installation environment of the game machine 100. When the information code is captured by the imaging unit 107, an image with reduced contrast overall or partially may be generated due to the influence of external light.

[0078] Furthermore, the maintenance state of the game machine 100 depends on the store where it is installed. If the maintenance state of the game machine 100 is poor, the printing of the information code by the card issuing unit 106 may be misaligned or blurred, or the lens or reflector of the imaging unit 107 may be dirty. For this reason, there is a possibility that noise may be superimposed on the captured image obtained by capturing the information code with the imaging unit 107.

[0079] Furthermore, the storage condition of the game card provided to the player depends on the length of time the game card has been held. If the game card has been held for a long time, the image of the printed information code may become blurred or dirty. Therefore, noise may be superimposed on the captured image obtained by capturing the information code with the imaging unit 107. Furthermore, if the game card is not placed correctly when capturing the image, shifting or geometric transformation may occur in the captured image obtained by capturing the information code with the imaging unit 107.

[0080] The game console 100 according to the fifth embodiment is equipped with an information code reading device 10, which converts the captured image into a reproduced image and reads the information indicated by the information code from the reproduced image after conversion. Therefore, even if the captured image contains various factors that hinder reading, the information indicated by the information code can be read well.

[0081] However, if the gaming console 100 has deteriorated significantly or if the game card provided to the player has deteriorated significantly, there is a possibility that reading of the information code will fail even if the captured image is converted into a reproduced image and the reproduced image after conversion is used. If reading of the information code fails, the gaming console 100 may notify the management server 200 of information indicating that fact.

[0082] 11 is a flowchart showing an example of a processing procedure in the game machine 100. When the control unit 101 of the game machine 100 acquires a captured image of an information code from the imaging unit 107 (step S501), the control unit 101 starts reading the information code with the information code reading device 10 (step S502). The information code reading device 10 converts the captured image into a reproduced image using the learning model MD, and executes a process of decoding the reproduced image after the conversion.

[0083] The control unit 101 determines whether the information code has been read successfully (step S503). If the control unit 101 obtains information indicating the information code from the information code reader 10, it determines that the reading has been successful, and if the control unit 101 does not obtain information indicating the information code, it determines that the reading of the information code has failed.

[0084] If it is determined that the information code has been successfully read (S503: YES), the control unit 101 refers to the attribute table and causes the character indicated by the information code to appear on the game screen, allowing the game to proceed (step S504). When the game is over, the control unit 101 terminates this flowchart. Furthermore, when the game is over (or when a predetermined clearing condition is met), the control unit 101 may control the card issuing unit 106 to provide the player with a game card created by printing the information code on a card mount.

[0085] If it is determined that reading of the information code has failed (S503: NO), the control unit 101 notifies the management server 200 of the failure to read the information code (step S505). The information notified to the management server 200 may include the identifier of the gaming machine 100, the store identifier of the store where the gaming machine 100 is installed, the reproduced image that failed to be read, and the captured image to be converted. The management server 200 may aggregate information notified from multiple gaming machines 100 installed in each store and notify the administrator of the aggregation result. Furthermore, based on the aggregation result, the management server 200 may suggest to the administrator improvements to the gaming machine 100, changes to the card mount, improvements to the installation environment, etc. Furthermore, the management server 200 may request an external server to re-learn the learning model MD. The learning model MD re-learned by the external server may be provided to the gaming machine 100 via communication and stored in the memory unit 12 of the information code reading device 10. The re-learning procedure is similar to the procedure for generating the learning model MD described in the third embodiment and the procedure for additional learning described in the fourth embodiment.

[0086] Furthermore, if the game machine 100 fails to read the information code, the game machine 100 may award a bonus to the player. Fig. 12 is a flowchart showing another example of the processing procedure in the game machine 100. The control unit 101 of the game machine 100 executes the same procedure as in the flowchart shown in Fig. 11, and determines whether or not the information code has been successfully read (steps S511 to S513). If it is determined that the information code has been successfully read (S513: YES), the control unit 101 proceeds with the game in the same procedure as described above (step S514).

[0087] If it is determined that the reading of the information code has failed (S513: NO), the control unit 101 grants a benefit to the player (step S515). The control unit 101, for example, controls the card issuing unit 106 to issue a new game card to replace the game card that has failed to be read, and provides the new game card to the player. Furthermore, if the game machine 100 is a game machine that reads multiple sets of game cards and causes characters corresponding to each set to appear and progress a scenario, and if the game machine fails to read some game cards but succeeds in reading the remaining game cards, the attribute of the character indicated by the successfully read information code may be changed (for example, strength may be increased by +1).

[0088] As described above, the game machine 100 according to the fifth embodiment progresses the game by reading the information code printed on the game card using the information code reader 10. When a captured image of an information code is input, the information code reader 10 converts the captured image into a reproduction image using a learning model MD that has been trained to output a reproduction image that reproduces the correct image of the information code, and reads the information indicated by the information code from the converted reproduction image. Therefore, even if the captured image contains a reading-impeding factor, the information indicated by the information code can be read well.

[0089] In the fifth embodiment, the gaming machine 100 is configured to include the information code reader 10, but the information code reader 10 may be connected to the outside of the gaming machine 100. Also, the management server 200 may be equipped with the information code reader 10, and the information code printed on the game card may be read via communication.

[0090] Furthermore, in the fifth embodiment, an example of application to a game machine has been described, but the present invention can also be applied to various devices equipped with a means for reading information codes, such as amusement machines that provide photo stickers to users, POS (Point of Sales) terminals, copiers, ID photo issuing machines, ticket vending machines, and ticket issuing machines.

[0091] The embodiments disclosed herein should be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0092] 10 Information code reader 11 Control section 12 Storage section 13 Input section 14 Output section PG1 Conversion Program PG2 reading program MD learning model MD1 generator MD2 discriminator

Claims

1. An information code reader provided in a game machine that issues game cards by printing information codes on card mounts, an acquisition unit that acquires a captured image of an information code printed on a game card issued by the game machine; a conversion unit that converts the captured image acquired by the acquisition unit into a reproduction of the correct image using a learning model that has been trained to output a reproduction image that reproduces the correct image of the information code when the captured image of the information code is input; a reading unit that reads information indicated by the information code from the reproduced image converted by the conversion unit; Equipped with The learning model is learned using a data set including processed images obtained by processing captured images or correct images as training data, The processed image includes an image with altered contrast Information code reader.

2. When the acquisition unit acquires a captured image, the conversion unit converts the captured image into a reproduced image; The reading unit executes a reading process for reading information indicated by the information code from the converted reproduced image.

2. The information code reader according to claim 1.

3. When the acquisition unit acquires the captured image, the reading unit executes a reading process to read information indicated by an information code from the captured image; If the reading unit fails to read the information, the conversion unit converts the captured image into a reproduction image, and the reading unit executes a reading process to read information indicated by the information code from the reproduction image after the conversion.

2. The information code reader according to claim 1.

4. The learning model is trained to output a reproduced image that reproduces the correct image of the information code when a captured image of the information code is input using a data set including a captured image of the information code and a correct image of the information code as training data.

4. An information code reader according to claim 1.

5. an issuing department that issues game cards by printing information codes on card mounts; an imaging unit that captures an image of the information code printed on the game card; an information code reader according to any one of claims 1 to 4, which reads an information code printed on a game card based on an image captured by the imaging unit; an execution unit that executes a game based on the information read by the information code reader; A game console equipped with:

6. a notification unit that notifies an external device of the fact that the information code reader has failed to read the information code printed on the game card when the information code reader has failed to read the information code printed on the game card; The gaming machine according to claim 5 , comprising:

7. a bonus granting unit that grants a bonus to the owner of the game card when the information code reading device fails to read the information code printed on the game card; 7. The gaming machine according to claim 5 or 6, comprising:

8. An information reading method executed by a game machine that issues game cards by printing information codes on card mounts, comprising: Acquire a captured image of an information code printed on a game card issued by the game machine; When a captured image of an information code is input, the acquired captured image is converted into a reproduced image of the correct image using a learning model that has been trained to output a reproduced image that reproduces the correct image of the information code; The information indicated by the information code is read from the converted reproduced image, The learning model is learned using a data set including processed images obtained by processing captured images or correct images as training data, The processed image includes an image with altered contrast How to read information codes.

9. A computer installed in a game machine that issues game cards by printing information codes on card mounts, Acquire a captured image of the information code; When a captured image of an information code is input, the acquired captured image is converted into a reproduced image of the correct image using a learning model that has been trained to output a reproduced image that reproduces the correct image of the information code; Output the converted reproduced image A computer program for executing a process, The learning model is learned using a data set including processed images obtained by processing captured images or correct images as training data, The processed image includes an image with altered contrast Computer program.

10. A data set including a plurality of pairs of captured images of the information code and correct images of the information code is acquired; Using the acquired data set as training data, a learning model is generated that outputs a reproduced image that reproduces the correct image of the information code when a captured image of the information code is input. A method for generating a learning model, comprising: The learning model is learned using a data set including processed images obtained by processing captured images or correct images as training data, The processed image includes an image with altered contrast How to generate a learning model.

11. The processed image further includes at least one of an image obtained by shifting the captured image or the correct image, an image obtained by geometric transformation, and an image obtained by adding noise. The method for generating a learning model according to claim 10.

Images