Information processing device, information processing method, and program

Abstract

To develop image processing techniques for the purpose of identification. [Solution] An information processing device 10 is provided, comprising: an acquisition unit 11 that acquires an image to be processed; a correction unit 12 that generates a processing image by adding a padding image with a width greater than or equal to a predetermined reference width around the image to be processed; and an image processing unit 13 that performs image processing on the processing image.

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Related techniques are disclosed in Patent Document 1. Patent Document 1 discloses a technique for determining the authenticity of a card medium.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An example of the object of this disclosure is to develop a technique for processing an image for the purpose of identification.

Means for Solving the Problems

[0005] According to one aspect of an example of this disclosure, a computer is caused to function as an acquisition unit that acquires a processing target image, a correction unit that generates a processing image by adding a padding image having a width equal to or greater than a predetermined reference width around the processing target image, an image processing unit that executes image processing on the processing image, by a program.

[0006] Also, according to one aspect of an example of this disclosure, one or more computers acquire a processing target image, generate a processing image by adding a padding image having a width equal to or greater than a predetermined reference width around the processing target image, An information processing method is provided that performs image processing on the aforementioned processing image.

[0007] Furthermore, according to one aspect of this example disclosure, A means for acquiring the image to be processed, Correction means for generating a processing image by adding a padding image with a width greater than or equal to a predetermined reference width around the image to be processed, Image processing means for performing image processing on the aforementioned processing image, An information processing device having the following is provided. [Effects of the Invention]

[0008] According to this example of disclosure, techniques for processing images for identification purposes can be developed. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of a functional block diagram of an information processing device. [Figure 2] Figure 2 is a flowchart showing an example of the processing flow of an information processing device. [Figure 3] Figure 3 shows an example of the hardware configuration of an information processing device. [Figure 4] Figure 4 is a diagram illustrating an example of a process performed by an information processing device. [Figure 5] Figure 5 is a diagram illustrating an example of the information processed by an information processing device. [Figure 6] Figure 6 illustrates another example of the information processed by the information processing device. [Figure 7] Figure 7 illustrates another example of the information processed by the information processing device. [Figure 8] Figure 8 is a diagram illustrating an example of the effects and benefits of an information processing device. [Figure 9] Figure 9 is a diagram illustrating another example of the effects and capabilities of an information processing device. [Figure 10]FIG. 10 is a diagram for explaining another example of the operational effects of the information processing apparatus. [Figure 11] FIG. 11 is a diagram for explaining another example of the operational effects of the information processing apparatus. [Figure 12] FIG. 12 is a diagram for explaining another example of the operational effects of the information processing apparatus. [Figure 13] FIG. 13 is a diagram for explaining another example of the operational effects of the information processing apparatus.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of this disclosure will be described with reference to the drawings. In this disclosure, the drawings are associated with one or more embodiments. Also, in all the drawings, the same reference numerals are assigned to the same components, and the description thereof will be omitted as appropriate.

[0011] <<First Embodiment>> FIG. 1 is a functional block diagram showing an overview of the information processing apparatus 10. FIG. 2 is a flowchart showing an example of the processing flow executed by the information processing apparatus 10.

[0012] As shown in FIG. 1, the information processing apparatus 10 includes an acquisition unit 11, a correction unit 12, and an image processing unit 13. The processing shown in the flowchart of FIG. 2 is executed by these functional units.

[0013] In S10, the acquisition unit 11 acquires a processing target image. In S11, the correction unit 12 generates a processing image from the processing target image acquired in S10. Specifically, the correction unit 12 generates a processing image by adding a padding image having a width equal to or greater than a predetermined reference width around the processing target image. In S12, the image processing unit 13 executes image processing on the processing image generated in S11.

[0014] Thus, the information processing device 10 does not process the image to be processed as is, but generates a processing image by adding a padding image with a width of "a predetermined standard width or more" around the image to be processed, and then processes that processing image. As will be described in detail later, the inventor has confirmed that when a sufficient amount of padding image (a padding image with a width of a predetermined standard width or more) is added around the image to be processed, the accuracy of identification by image processing improves. The information processing device 10 that performs the above-described processing can improve the accuracy of identification by image processing.

[0015] Incidentally, in the YOLO (You Only Look Once) model, which is an example of an image processing model, "letterboxing" is standardly employed to adjust the input image to a specified size required by the model (for example, 640 x 640 pixels). This involves scaling the input image while maintaining its aspect ratio (the ratio of vertical length to horizontal length; the same applies hereafter) and filling the missing area with black (zero value). Although letterboxing also involves adding a black area around the input image, the process of adding a padding image performed by the information processing device 10 differs from letterboxing, and the resulting effects are also different.

[0016] Specifically, as described above, the information processing device 10 adds padding images with a width "greater than or equal to a predetermined standard width" around the image to be processed. In one example, the width of the padding image is predetermined. In another example, the lower limit of the width of the padding image is predetermined. The information processing device 10 adds padding images with a width "greater than or equal to a predetermined standard width" around the image to be processed by adding padding images based on these predetermined rules. In the case of processing by such an information processing device 10, regardless of what kind of image to be processed is being processed, the generated processing image will always have padding images with a width "greater than or equal to a predetermined standard width" around the image to be processed. As described above, the inventor has confirmed that when a sufficient amount of padding image (a padding image with a width "greater than or equal to a predetermined standard width") is added around the image to be processed, the accuracy of identification by image processing is improved. With the information processing device 10 that can add padding images with a width "greater than or equal to a predetermined standard width" around the image to be processed, the accuracy of identification by image processing can be improved regardless of what kind of image to be processed is being processed. In other words, it is possible to suppress the inconvenience of variations in the accuracy of image processing for each image being processed. Furthermore, it is possible to maintain a consistently high level of accuracy in image processing for identification.

[0017] On the other hand, in letterbox processing, the width and lower limit of the black area to be added are not predetermined. Letterbox processing enlarges or reduces the input image while maintaining its aspect ratio, and then fills the missing area with black (zero value) when compared to a predetermined size with a specified aspect ratio. The width of the added black area is determined based on the result of this comparison (the missing area). If the missing area is large, the width of the black area will be large, and if the missing area is small, the width of the black area will be small. Also, it is possible that no black area will be added, such as when the aspect ratio of the input image matches the aspect ratio of the predetermined size. Thus, the processing performed in letterbox processing does not add a black area that is greater than a predetermined standard width. In the case of such letterbox processing, the width of the added black area may differ depending on the aspect ratio of the input image, its size, and the size after enlargement or reduction.

[0018] The basic idea behind letterboxing is to make the input image as large as possible and the added black area as small as possible. In other words, in letterboxing, it is preferable for the width of the added black area to be small. Letterboxing, based on this idea, does not pre-determine a standard width for the added black area and add a black area wider than or equal to that standard width. In such letterboxing, the width of the added black area can vary greatly from one input image to another. As a result, the accuracy of recognition by image processing can also vary.

[0019] <<Second Embodiment>> <Overview> The information processing device 10 of the second embodiment is a concrete implementation of the configuration of the information processing device 10 of the first embodiment.

[0020] Deep neural networks, known as YOLO, have gained widespread popularity due to their versatility in handling a wide range of image and object recognition machine learning tasks. In particular, compared to recent large language models (LLMs), YOLO is lightweight and can run on edge devices such as smartphones, enabling the practical application of extracting various information in real time from images captured using smartphone cameras. However, YOLO has difficulty accurately identifying the language of characters written in images, and there has been a particular challenge in that the recognition accuracy is significantly lower for characters printed using special printing techniques such as gold foil stamping or hologram processing compared to characters printed with regular black ink.

[0021] As described in the first embodiment, the information processing device 10 of this embodiment does not process the image to be processed as is, but rather generates a processing image by adding a padding image with a width of "a predetermined reference width or greater" around the image to be processed, and can then process that processing image with a YOLO model. This significantly improves the accuracy of recognition by image processing, for example, the recognition accuracy of a small YOLO model that can be installed in a smartphone or the like. Note that the image processing model used in this embodiment is not limited to a YOLO model.

[0022] <Hardware Configuration> Next, an example of the hardware configuration of the information processing device 10 will be described. Each functional unit of the information processing device 10 is realized by any combination of hardware and software. It will be understood by those skilled in the art that there are various variations in the implementation method and the device. The software includes programs that are pre-installed at the time of shipment of the device, as well as programs downloaded from recording media such as CDs (Compact Discs) or from servers on the Internet.

[0023] Figure 3 is a block diagram illustrating the hardware configuration of the information processing device 10. As shown in Figure 3, the information processing device 10 includes a processor 1A, memory 2A, input / output interface 3A, peripheral circuitry 4A, and bus 5A. Peripheral circuitry 4A includes various modules. The information processing device 10 does not necessarily have peripheral circuitry 4A. The information processing device 10 may also be composed of multiple physically and / or logically separated devices. In this case, each of the multiple devices may have the above hardware configuration.

[0024] Bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuits 4A, and input / output interface 3A to send and receive data to and from each other. Processor 1A is a processing unit such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit). Memory 2A is a memory such as RAM (Random Access Memory) or ROM (Read Only Memory). Input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. Input / output interface 3A also includes interfaces for connecting to communication networks such as the Internet. Input devices include, for example, keyboards, mice, microphones, physical buttons, touch panels, etc. Output devices include, for example, displays, projectors, speakers, printers, mailers, etc. Processor 1A can issue commands to each module and perform calculations based on their calculation results.

[0025] <Functional Configuration> Next, the functional configuration of the information processing device 10 will be described in detail. In this embodiment, the information processing device 10 is a user terminal operated by the user. Examples of user terminals include, but are not limited to, smartphones, tablet devices, personal computers, game consoles, mobile phones, smartwatches, and other wearable devices.

[0026] Figure 1 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an acquisition unit 11, a correction unit 12, and an image processing unit 13. These functional units execute the processes shown in the flowchart of Figure 2. Each process will be explained below.

[0027] (S10: Step to acquire the image to be processed) The process in S10 is performed by the acquisition unit 11. The acquisition unit 11 acquires the image to be processed.

[0028] The "image to be processed" is an image that is to be processed by the image processing unit 13. Image processing includes the process of determining and classifying what the objects contained in the image are, i.e., identification processing. The image to be processed may be a photograph taken with a camera, an image generated by an image generation application / software, an image generated by scanning with a scanner, an image generated by taking a screenshot, or any other image. Furthermore, the image to be processed may be an image generated by applying a predetermined processing method to any of these images. The processing method may be, but is not limited to, cropping a part of the image.

[0029] The content of the images to be processed varies. The images may include objects, text, numbers, symbols, and even people or characters.

[0030] There are various means of acquiring the image to be processed. In one example, the acquisition unit 11 activates the camera function of the information processing device 10 and has the user photograph a predetermined subject. The acquisition unit 11 then acquires the captured image generated from the photograph. In this example, the acquisition unit 11 may acquire the captured image as the image to be processed. Alternatively, the acquisition unit 11 may acquire an image generated by applying a predetermined processing to the captured image as the image to be processed. For example, the acquisition unit 11 may cut out a portion of the captured image (the portion showing the predetermined subject) and acquire the cut-out image as the image to be processed.

[0031] In another example, the information processing device 10 reads an image stored in a predetermined storage device in response to a user operation. In this example, the acquisition unit 11 may acquire the read image as the image to be processed. Alternatively, the acquisition unit 11 may acquire an image generated by applying a predetermined processing operation to the read image as the image to be processed. For example, the acquisition unit 11 may cut out a portion (the portion containing a predetermined subject) from the read image and acquire the cut-out image as the image to be processed. The predetermined storage device may be provided inside the information processing device 10, or it may be provided in an external device that is communicably connected to the information processing device 10. The predetermined storage device may have images taken with a camera, images generated by image generation applications / software, images generated by scanning with a scanner, images generated by screenshots, etc., stored in advance.

[0032] Here, we will explain a specific example of the processing performed by the acquisition unit 11. Note that this is just one example and is not limited to this.

[0033] In this specific example, the information processing device 10 is used in a card game. Various information related to the game is displayed on the cards. For example, a card may display a picture of a character. A card may also display the character's name and various information about the character. A card may also display a handwritten-style signature. At least some cards may have special processing to give them a special feel. For example, at least some cards may have at least one of holographic processing and foil stamping (such as gold foil stamping). For example, at least some cards may display text that has at least one of holographic processing and foil stamping. Furthermore, each of multiple cards may display characters in a specific language (such as English, Japanese, or Simplified Chinese). Among the multiple cards, there may be cards that display character information in different languages.

[0034] The user (player) playing the game uses the information processing device 10 to photograph the card. Figure 4 schematically shows an example of the photographing process. Figure 4 shows the process of photographing card C (subject) using the information processing device 10. In Figure 4, the display of the information processing device 10 functions as a viewfinder, and card C, which is within the shooting range of the camera of the information processing device 10, is displayed in the viewfinder.

[0035] The acquisition unit 11 can display a frame W on the viewfinder (the display of the information processing device 10). Preferably, the aspect ratio of the frame W is the same as or equivalent to the aspect ratio of the card C. The user photographs the card C so that it fits within the frame W, using a size equivalent to the frame W. Incidentally, the illustrated card C displays a picture of a character. The card C also displays the character's name (letters N) and a handwritten-style signature (signature S). The character's name may be displayed in a specific language (English, Japanese, Simplified Chinese, etc.). The letters on the card C may also be foil-stamped and have at least one of the hologram processing applied.

[0036] The acquisition unit 11 acquires the captured image in this manner. The captured image to be acquired may be of a fixed size (vertical and horizontal dimensions are fixed; the same applies hereinafter) or of a variable size (vertical and horizontal dimensions are variable; the same applies hereinafter). The acquisition unit 11 then crops a portion of the captured image and acquires the cropped image as the image to be processed. In one example, the acquisition unit 11 can crop the image inside the frame W and acquire the cropped image as the image to be processed. In this example, the acquisition unit 11 can crop an image to be processed of a predetermined fixed size (size of the frame W). In another example, the acquisition unit 11 can process the captured image to detect the area in which card C is visible (the area of ​​the object in the captured image), crop the detected area in which card C is visible, and acquire the cropped image as the image to be processed. In this example, the image to be processed acquired by the acquisition unit 11 may be of a variable size.

[0037] The detection of the area containing card C can be achieved using widely known image processing techniques. For example, the acquisition unit 11 may detect the area containing card C using a template matching algorithm (such as AKAZE: Accelerated-KAZE). Alternatively, the acquisition unit 11 may detect object areas by extracting the contours of objects in the image using edge detection. The acquisition unit 11 may then detect object areas where the similarity to the contour shape of card C, which is stored in advance, is above a threshold as the area containing card C. In addition to the contour shape, other features of the appearance of card C may be used to detect the area containing card C from among the detected object areas. The processing used to detect the area containing card C is not limited to this example.

[0038] (S11: Process of generating a processing image from the image to be processed) The process in S11 is performed by the correction unit 12. The correction unit 12 generates a processing image from the image to be processed acquired by the acquisition unit 11. Specifically, the correction unit 12 generates a processing image by adding a padding image with a width greater than or equal to a predetermined reference width around the image to be processed.

[0039] First, Figures 5 to 7 will be used to explain the image to be processed and the image used for processing.

[0040] Figure 5 shows the image to be processed acquired by the acquisition unit 11. The vertical length of the image to be processed is defined as Ho, and the horizontal length of the image to be processed is defined as Wo. The values ​​of Ho and Wo may be the same for all images to be processed, or they may be different for each image to be processed.

[0041] For example, if a configuration is adopted in which a frame W (see Figure 4) is displayed in the viewfinder as described above, and the image inside frame W is cropped to generate the image to be processed, the values ​​of Ho and Wo will be the same for all images to be processed. In other words, the size of the image to be processed will be fixed. On the other hand, if a configuration is adopted in which, for example, an object region in an image is detected, the detected object region is cropped, and the cropped image is acquired as the image to be processed, the values ​​of Ho and Wo may differ for each image to be processed, depending on, for example, how the user takes the picture or how the user generates the image. In other words, the size of the image to be processed will be variable.

[0042] Figure 6 shows the image size (processing size) defined for the image processing performed by the image processing unit 13, which will be described later. The image processing unit 13 can process images using a learned model. The size of the image to be processed by this learned model is defined in advance as the processing size. The vertical length of the processing size is defined as Hs, and the horizontal length of the processing size is defined as Ws.

[0043] Wo and Ws may or may not be the same. Similarly, Ho and Hs may or may not be the same.

[0044] Figure 7 shows the processing image generated by the correction unit 12. The vertical length of the processing image is Hs, and the horizontal length of the processing image is Ws. In other words, the vertical and horizontal lengths of the processing image match the vertical and horizontal lengths of the processing size shown in Figure 6.

[0045] As shown in Figure 7, the image to be processed is located in the center of the processing image. A padding image with a predetermined width of P or more is added around the image to be processed, surrounding it. The widths of the padding images added above and below the image to be processed may be the same as or different from the widths of the padding images added to the left and right of the image to be processed. The padding image is preferably a black-filled image, as shown in Figure 7. The image to be processed located in the center of the processing image may be at its original size (Wo × Ho), enlarged from its original size, or reduced from its original size. If the area of ​​the processing image is X, the area of ​​the image to be processed within the processing image is preferably between 0.4X and 0.8X. By adding a sufficient amount of padding around the image to be processed so as to satisfy this relationship, the accuracy of recognition by image processing is improved.

[0046] The reference width P is a common value for all images to be processed. In other words, the information processing device 10 can acquire multiple images to be processed with a time difference, generate processing images for multiple images to be processed, and perform image processing, but it can generate processing images by applying the same reference width to multiple images to be processed.

[0047] By the way, the processing size is fixed. And, in one example, the size of the image to be processed is fixed. In this case, the width of the padding image is also fixed. The correction unit 12 can generate the processing image by adding a padding image of a predetermined fixed width (e.g., reference width P) around the fixed-size image to be processed. In this example, the reference width P defines the width of the padding image.

[0048] In another example, the processing size is fixed, but the size of the image to be processed is variable. In this case, the width of the padding image is also variable. The correction unit 12 can then generate the processing image by adding a padding image with a width (variable) of at least a predetermined reference width P around the fixed-size image to be processed. In this example, the reference width P defines the lower limit of the padding image width.

[0049] Next, an example of the process for generating an image for processing will be described. The correction unit 12 can execute either processing example 1 or 2 below. "Processing Example 1" Processing Example 1 is primarily used when the image to be processed has a variable size. However, this processing may also be performed when the image to be processed has a fixed size.

[0050] The correction unit 12 can perform (1) size adjustment of the image to be processed, (2) alignment of the image to be processed within the processing image, and (3) application of padding.

[0051] (1) Adjusting the size of the image to be processed Here, the correction unit 12 determines the size of the image to be processed within the processing image. The correction unit 12 can adjust the size of the image to be processed to the optimal size for inclusion in the processing image while maintaining the aspect ratio of the image to be processed. The options here are: (A) use the original size of the image to be processed, (B) reduce the size of the image to be processed while maintaining the aspect ratio, and (C) enlarge the image to be processed while maintaining the aspect ratio.

[0052] First, as described above using Figures 5 to 7, the various parameters are defined as follows. • Horizontal length of the image to be processed: Wo • Vertical length of the image to be processed: Ho • Horizontal length of processing size: Ws • Vertical length of processing size: Hs ·Standard width: P

[0053] The correction unit 12 can adopt the original size of the image to be processed if Ho matches (Hs-2P) and Wo matches (Ws-2P) (selection condition 1). Note that the match here may be an exact match. In addition, the concept of a match here may include an exact match and a state in which the difference is within a predetermined threshold. 2P is the sum of the reference widths of the padding images in the entire horizontal direction. As shown in Figure 7, padding images with a width of P or more are added to both the right and left sides of the image to be processed. Therefore, the sum of the reference widths of the padding images in the entire horizontal direction is 2P.

[0054] Furthermore, if selection condition 1 is not met and Wo × (Hs-2P) / Ho ≤ (Ws-2P) (selection condition 2), the correction unit 12 can adopt (Hs-2P) / Ho as the magnification or reduction ratio of the image to be processed. That is, the correction unit 12 uses the inequality to determine whether the length after magnification / reduction will be less than or equal to the value obtained by subtracting 2P from the horizontal length of the processing size Ws when the magnification or reduction ratio is adopted to match the vertical length Ho of the image to be processed to the value obtained by subtracting 2P from the vertical length Hs of the processing size. If the inequality is met, the correction unit 12 adopts the magnification or reduction ratio that matches the vertical length Ho of the image to be processed to the value obtained by subtracting 2P from the vertical length Hs of the processing size. The correction unit 12 can magnify or reduce the image to be processed at the determined magnification or reduction ratio while maintaining the aspect ratio of the image to be processed. Enlarging or reducing the size of the image being processed can be achieved using widely known technologies such as OpenCV.

[0055] Furthermore, if the selection condition 1 is not met and the above inequality (selection condition 2) is not met, i.e., if Wo × (Hs-2P) / Ho > (Ws-2P) (selection condition 3), the correction unit 12 adopts (Ws-2P) / Wo as the magnification or reduction ratio of the image to be processed. The magnification or reduction ratio adopted in this case is the ratio that makes the "horizontal length Wo of the image to be processed" match the "horizontal length Ws of the processing size minus 2P". The correction unit 12 can enlarge or reduce the image to be processed at the determined magnification or reduction ratio while maintaining the aspect ratio of the image to be processed. Enlarging or reducing the image to be processed can be achieved using widely known technologies such as OpenCV.

[0056] In this way, the correction unit 12 can determine whether each image to be processed satisfies any of the selection conditions 1 to 3. Then, the correction unit 12 can perform a single process corresponding to the selected condition that is satisfied to adjust the size of the image to be processed.

[0057] (2) Alignment of the image to be processed within the processing image Here, the correction unit 12 determines the position of the image to be processed within the processing image. It is preferable to position the image to be processed at the center of the processing image. The correction unit 12 executes a process that can achieve such a position. There are various algorithms for achieving such a position. For example, the correction unit 12 may determine the position of the image to be processed within the processing image such that the center point of the processing image (center point of the rectangle) and the center point of the image to be processed (center point of the rectangle) overlap, and their longer sides are parallel. It is preferable that the center point of the processing image and the center point of the image to be processed completely overlap, but a slight misalignment is acceptable. Even in such cases, the effects of this embodiment can be achieved.

[0058] (3) Add padding image The correction unit 12 places the image to be processed, whose size is determined by the above process, at the position determined within the processing image, and then adds padding to the blank areas (areas where the image to be processed does not exist) within the processing image.

[0059] In the above-mentioned (1) size adjustment of the image to be processed, if Ho matches (Hs-2P) and Wo matches (Ws-2P) (selection condition 1), the correction unit 12 adopts the original size of the image to be processed. In this case, the correction unit 12 can generate a processing image of the processing size by adding padding images of width P to the top and bottom and left and right sides of the image to be processed.

[0060] Furthermore, in the above-mentioned (1) size adjustment of the image to be processed, if selection condition 1 is not met and Wo × (Hs-2P) / Ho ≤ (Ws-2P) (selection condition 2), the correction unit 12 adopts (Hs-2P) / Ho as the magnification or reduction ratio of the image to be processed. In this case, the correction unit 12 adds padding images with a width of P to the top and bottom of the image to be processed after magnification or reduction. The correction unit 12 also adds padding images with a width of (Ws-Wo × (Hs-2P) / Ho) / 2 to the left and right of the image to be processed after magnification or reduction. Through this process, the correction unit 12 can generate a processing image of the size required for processing.

[0061] Furthermore, in the above-mentioned (1) size adjustment of the image to be processed, if selection condition 1 is not met and Wo × (Hs-2P) / Ho > (Ws-2P) (selection condition 3), the correction unit 12 adopts (Ws-2P) / Wo as the magnification or reduction ratio of the image to be processed. In this case, the correction unit 12 adds padding images with a width of P to the left and right sides of the image to be processed after magnification or reduction. The correction unit 12 also adds padding images with a width of (Hs-Ho × (Ws-2P)) / Wo) / 2 to the top and bottom sides of the image to be processed after magnification or reduction. Through this process, the correction unit 12 can generate a processing image of the size required for processing.

[0062] "Processing Example 2" Processing Example 2 is primarily used when the image to be processed has a fixed size. In this example, the processing size, the size of the image to be processed, and the width of the padding image are all fixed sizes and predetermined. The position of the image to be processed within the processing image is also predetermined. The correction unit 12 generates the processing image by placing the image to be processed at a predetermined position within the processing image and adding a padding image of a predetermined fixed width around it.

[0063] As a variation of processing example 2, the scaling ratio of the image to be processed may be predetermined. The correction unit 12 may then enlarge or reduce the image to be processed by that scaling ratio while maintaining the aspect ratio, then place the enlarged or reduced image to be processed at a predetermined position within the processing image, and generate the processing image by adding a padding image of a predetermined fixed width around it.

[0064] Here, a modified version of the process for generating the processing image will be described. In the process described above, the correction unit 12 places the resized image to be processed at a predetermined position within the processing image, and then adds a padding image to the margins around the image to be processed (the parts of the processing image where the image to be processed does not exist).

[0065] As a variation, the correction unit 12 may prepare a processing image in advance in which all pixels have a zero value (black, RGB(0,0,0)). Then, the correction unit 12 may generate a processing image by drawing the resized image to be processed on top of (overlaying) it at a predetermined position within the processing image, thereby adding a padding image around the image to be processed.

[0066] (S12: Step of performing image processing on the image to be processed) The process in S12 is performed by the image processing unit 13. The image processing unit 13 performs image processing on the processing image generated by the correction unit 12.

[0067] The content of image processing varies. For example, the image processing unit 13 can input an image for processing into a learning model such as a YOLO model and obtain the result. Note that the learning model is not limited to a YOLO model. The configuration of the learning model varies. The learning model can detect the region of a predetermined object in an image. The learning model may detect the region of a common object such as a person or a vehicle in an image. In addition, the learning model may detect characteristic regions during learning through individual learning. For example, the learning model may detect the region of an object with a characteristic appearance in an image by learning images of an object with a characteristic appearance. Alternatively, the learning model may detect the region of a character with a characteristic appearance in an image by learning images of a character with a characteristic appearance (such as a handwritten signature, a character with special processing such as foil stamping or hologram processing, or a character in a specific language). The learning model may be implemented within the information processing device 10, or it may be implemented in an external device (e.g., a server) that is communicatively connected to the information processing device 10.

[0068] Here, we will explain a specific example of image processing performed by the image processing unit 13. Note that this is merely one example and is not limited to this example.

[0069] In this specific example, the information processing device 10 is used in a card game. The cards display various information related to the game. For example, the cards may display images of characters. The cards may also display the names of characters and text indicating various information about the characters. Furthermore, the cards may display handwritten-style signatures. In addition, at least some of the cards may have special processing to give them a sense of exclusivity. For example, at least some of the cards may have at least one of holographic processing and foil stamping.

[0070] The image processing unit 13 processes the processing image containing such a card using the YOLO model. The output of the YOLO model includes at least one of the following: • Card type (Card ID: Identifier) • Whether or not there is a handwritten-style signature and where it is located • The position of the characters indicating the character's name, and the language in which they are written (English, Japanese, Simplified Chinese, etc.)

[0071] <Effects and Effects> According to the information processing device 10 of this embodiment, the same effects and advantages as those of the information processing device 10 of the first embodiment are achieved. That is, the information processing device 10 generates a processing image in which a padding image with a width greater than or equal to a predetermined reference width is added around the image to be processed, and by processing this processing image, the accuracy of identification by image processing can be improved.

[0072] In one example, the information processing device 10 can detect a predetermined object region (e.g., a card region) within an image, crop the detected object region, and generate a processing image by adding a padding image around the cropped image (the image to be processed). The generated processing image can then be processed using the YOLO model. By cropping a portion of the image and then adding a padding image around the cropped portion, the information processing device 10 can add a sufficient amount of padding. As another means of adding a sufficient amount of padding, one could consider reducing the size of the image without cropping a portion of it, and then adding a padding image around the reduced image. However, in this case, the target object region within the image becomes smaller, which may reduce the accuracy of identification by image processing. By cropping the target object region within the image and then adding a padding image around it, it is possible to add a sufficient amount of padding while keeping the target object region at a sufficient size.

[0073] Incidentally, the YOLO model takes the entire image as input and detects the position (bounding box) and class of all objects present in the image at once. This is the origin of the name YOLO, which stands for "You Only Look Once," and is the reason for its high speed. When using a YOLO model with such features, the process becomes redundant if you detect a specific object region in the image (e.g., a card region) and perform processing such as cropping the detected object region before processing with the YOLO model. For this reason, this method of use is not usually employed. Furthermore, the purpose of discrimination models such as YOLO is to estimate the "class (or label)" to which each object in an image belongs. Here, "class (or label)" refers to a discrete discrimination unit that constitutes a set of categories used by the model to classify objects in an image. For example, strings such as "person," "car," "dog," and "traffic light" are classes, and each is learned by the model as one of the predefined classes. Technically, a class is an element of the domain of the probability distribution given as output for objects in the input image. The model takes an image as input and estimates the corresponding class along with the probability of an object's presence in each candidate region.

[0074] To demonstrate the effects of the information processing device 10, the results of the evaluation experiment are shown in Figures 8 to 13. Hereafter, the process by which the information processing device 10 adds the aforementioned padding image to improve the accuracy of image processing-based identification will be referred to as CBP (Context-Boost Padding).

[0075] Figures 8 through 13 show a comparison of the accuracy of YOLO inference performed without CBP and with CBP, using template matching (such as AKAZE) to extract the card portion from images of cards taken with a smartphone. The same YOLO model was used in both the YOLO inference without CBP and with CBP, with the difference being the presence or absence of CBP in the input data.

[0076] Multiple cards were used for evaluation. As shown in Figure 4, card C displayed a character's image. Among the multiple cards, some displayed different character images.

[0077] Furthermore, card C displayed the letter N, which represents the character's name. Among the various cards, some had at least one of the following processes applied to the letter N: hologram processing and / or foil stamping (such as gold foil stamping), while others did not. In addition, among the various cards, some displayed the character's name in different languages ​​(English, Japanese, Simplified Chinese, etc.).

[0078] Furthermore, the cards included both those with a handwritten-style signature "S" and those without. The handwritten-style signature "S" had at least one of the following processes applied: hologram processing and foil stamping (such as gold foil stamping).

[0079] YOLO inference estimated the following: • Card type (card ID) • Whether or not there is a handwritten-style signature and where it is located • The position of the characters indicating the character's name, and the language in which they are written (English, Japanese, Simplified Chinese, etc.)

[0080] Figure 8 shows the accuracy of determining the card type (card ID). Figure 8 shows the percentage (SCORE) of correct card IDs output by the YOLO model.

[0081] Figure 9 shows the signature detection accuracy. The signature detection accuracy was calculated using the following formulas, based on the number of cards with printed signatures that were correctly identified (true positives) TP, the number of cards with printed signatures that were not identified (false negatives) FN, the number of cards without printed signatures that were incorrectly identified (false positives) FP, and the number of cards without printed signatures that were incorrectly identified (true negatives) TN.

[0082]

number

[0083]

number

[0084]

number

[0085]

number

[0086]

number

[0087] Accuracy represents the overall accuracy rate. Positive Precision represents the percentage of signatures detected by the YOLO model that were correct. Negative Precision represents the percentage of signatures that were not detected by the YOLO model that were correct. Recall represents the percentage of cards with printed signatures that were able to have their signatures detected. Specificity represents the percentage of cards without printed signatures that were not able to have their signatures detected.

[0088] Figures 10 to 13 show the language determination accuracy. The accuracy of language determination was calculated using the following formulas, based on the number of cards identified as language L (Japanese, English, or Chinese) (true positives) TP, the number of cards identified as not being language L (false negatives) FN, the number of cards not being language L identified as language L (false positives) FP, and the number of cards not being language L identified as not being language L (true negatives) TN.

[0089]

number

[0090]

number

[0091]

number

[0092] Precision represents the percentage of the YOLO model correctly identified as language L. Recall represents the percentage of language L cards that were correctly identified as language L. The F-score (F1) represents the harmonic mean of Precision and Recall. The weighted average F-score, a combined metric for all three languages, is calculated by multiplying the ratio of Japanese, English, and Chinese cards in the test data by the F-score for each language and summing them up.

[0093] These results show that YOLO inference is more accurate with CBP than without it. In particular, with CBP, highly accurate identification is possible even for cards that have only very slight differences between the Japanese and English versions. Although the mechanism by which applying padding images to inference images improves inference performance has not yet been clearly identified, we propose the following hypothesis.

[0094] "Clarification of image topology information" By explicitly representing the black padding image as a zero-value region, the positional information of the card region being inferred becomes clearer, which could potentially be effective in extracting features from the card itself.

[0095] "Instructional effect of background information removal" It's possible that background and other peripheral information acquired during the training phase is being encouraged to be recognized as noise to be ignored during the inference phase. Since black padding images are explicitly indicated with a zero value as "regions not to be inferred," the network may be induced to ignore background information.

[0096] <Examples> Here, an embodiment of the information processing device 10 will be described. Note that this embodiment is merely an example, and the embodiments of the information processing device 10 are not limited to this example.

[0097] The information processing device 10 is used in a card game. The information processing device 10 is a user terminal operated by the user (game player). Examples of user terminals include, but are not limited to, smartphones, tablet devices, personal computers, game consoles, mobile phones, smartwatches, and other wearable devices.

[0098] The user (game player) takes a picture of the card using the camera function of the information processing device 10. Details are as described above with reference to Figure 4. The information processing device 10 acquires the captured image. The captured image acquired here may be of a fixed size.

[0099] Next, the information processing device 10 crops a portion of the captured image and acquires the cropped image as the image to be processed. The image to be processed generated by cropping here may be of a fixed size. For example, the information processing device 10 may display a frame W on the display that guides the card position and size during shooting. Then, the information processing device 10 may crop the image inside this frame W from the captured image and acquire the cropped image as the image to be processed. The frame W may be of a fixed size. In this case, the image inside the frame W (the image to be processed) will also be of a fixed size.

[0100] The information processing device 10 then generates a processing image from the image to be processed and processes the processing image using the YOLO model. These processes performed by the information processing device 10 are as described above.

[0101] The information processing device 10 uses the output results of the YOLO model as at least some of the game's parameters to advance the game. The content of the game varies.

[0102] <Variation> In the second embodiment, the information processing device 10 was a user terminal. As a variation, the information processing device 10 may be a server. In this case, the information processing device 10 may acquire an image from a client terminal and process the image in the manner described above.

[0103] Although this disclosure has been described above with reference to embodiments, this disclosure is not limited to the embodiments described above. Various modifications to the structure and details of this disclosure are possible, which can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0104] Furthermore, the flowchart used in the above explanation shows multiple steps (processes) in sequence. However, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content.

[0105] Some or all of the above embodiments may also be described as follows, but are not limited to the following. 1. Computers, means for acquiring an image to be processed, Correction means for generating a processing image by adding a padding image with a width greater than or equal to a predetermined reference width around the image to be processed. Image processing means for performing image processing on the aforementioned processing image, A program that makes it function as such. 2. The correction means is, The program according to claim 1, which generates the processing image by adding the padding image of a predetermined fixed width around the processing target image. 3. The acquisition means is, The process involves detecting an object region within an image, cropping the detected object region, and acquiring the cropped image as the image to be processed. The aforementioned image processing means is A program according to 1 or 2 that processes the processing image using the YOLO model. 4. The acquisition means is, Obtain an image of a fixed size, From the fixed-size image, a predefined fixed-size image to be processed is extracted. The correction means is A program according to any one of 1 to 3 that generates the processing image by adding a padding image of a predetermined fixed width around the fixed-size image to be processed. 5. The correction means is a program according to any one of 1 to 4 that generates the processing image by applying the same reference width to a plurality of images to be processed. 6. The correction means is, After enlarging or reducing the image to be processed, a padding image with a width equal to or greater than the reference width is added around the enlarged or reduced image to be processed, thereby generating the processing image of the processing size. The horizontal length of the image to be processed is Wo, The vertical length of the image to be processed is Ho, The lateral length of the processing size is Ws, The vertical length of the processing size is Hs, Let the aforementioned reference width be P. In that case, The correction means is If Wo × (Hs-2P) / Ho ≤ (Ws-2P), then (Hs-2P) / Ho is adopted as the magnification or reduction ratio of the image to be processed. If Wo × (Hs-2P) / Ho > (Ws-2P), then the program described in any of 1 to 5 adopts (Ws-2P) / Wo as the magnification or reduction ratio of the image to be processed. 7. When the correction means adopts (Hs-2P) / Ho as the magnification or reduction ratio of the image to be processed, The program according to 6 generates a processing image of the processing size by adding a padding image with a width of P to the top and bottom of the processing target image after it has been enlarged or reduced, and adding a padding image with a width of (Ws-Wo×(Hs-2P) / Ho) / 2 to the left and right of the processing target image after it has been enlarged or reduced. 8. When the correction means adopts (Ws-2P) / Wo as the magnification or reduction ratio of the image to be processed, A program according to 6 or 7 that generates a processing image of the processing size by adding a padding image with a width of P to the left and right sides of the processing target image after it has been enlarged or reduced, and adding a padding image with a width of (Hs-Ho×(Ws-2P)) / Wo) / 2 to the top and bottom sides of the processing target image after it has been enlarged or reduced. 9. The program according to any one of 1 to 8, wherein the image to be processed is an image that includes a subject on which at least one of hologram processing and foil stamping has been performed. 10. One or more computers, Obtain the image to be processed, A processing image is generated by adding a padding image with a width greater than or equal to a predetermined standard width around the image to be processed. An information processing method that performs image processing on the aforementioned image for processing. 11. Acquisition means for acquiring the image to be processed, Correction means for generating a processing image by adding a padding image with a width greater than or equal to a predetermined reference width around the image to be processed, Image processing means for performing image processing on the aforementioned processing image, An information processing device having

[0106] Some or all of the appendices 2 through 9, which are dependent on the program described in appendice 1 above, may also be dependent on the information processing method in appendice 10 and the information processing device in appendice 11, in the same dependent relationship as between appendice 1 and appendices 2 through 9. Furthermore, without departing from the embodiments described above, some or all of the configurations described as appendices can be realized in various hardware, software, various recording means for recording software, or systems. [Explanation of Symbols]

[0107] 10 Information Processing Devices 11 Acquisition Department 12 Correction section 13 Image Processing Unit 1A Processor 2A Memory 3A input / output I / F 4A Peripheral Circuits 5A Bus

Claims

1. Computers, Acquisition means for acquiring an image and cropping a predefined fixed-size image to be processed from the acquired image, Correction means that superimposes the image to be processed at a predetermined position within a predefined fixed-size processing image in which all pixels are zero, thereby adding a padding image of a predetermined fixed width around the image to be processed. Image processing means for performing image processing on the processing image obtained by superimposing the aforementioned image to be processed, A program that makes it function as such.

2. The program according to claim 1, wherein the image to be processed is an image that includes a subject on which at least one of hologram processing and foil stamping has been performed.

3. One or more computers, An image is acquired, and a predefined fixed-size image to be processed is cropped from the acquired image. By superimposing the image to be processed at a predetermined position within a predefined fixed-size processing image in which all pixels are set to zero, a padding image of a predetermined fixed width is added around the image to be processed. An information processing method that performs image processing on the processing image obtained by superimposing the aforementioned image to be processed.

4. Acquisition means for acquiring an image and cropping a predefined fixed-size image to be processed from the acquired image, Correction means that superimposes the image to be processed at a predetermined position within a predefined fixed-size processing image in which all pixels are zero, thereby adding a padding image of a predetermined fixed width around the image to be processed. Image processing means for performing image processing on the processing image obtained by superimposing the aforementioned image to be processed, An information processing device having

Images