Program, information processing method, and information processing device

Abstract

To provide a program, an information processing method, and an information processing device capable of acquiring a status of a game.SOLUTION: A program causes a computer to execute an information processing method for detecting a position of a card in an image captured by a camera of a range where a card for a game is arranged, generating a cutout image by cutting out a region of the card from the image on the basis of the position of the card, and acquiring a status of the game on the basis of the cutout image.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present technology relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] In recent years, trading card games (TCGs) have become popular. TCGs are card games in which multiple players compete against each other according to predetermined rules using multiple cards with various different pictures and characters printed on them.

[0003] To allow players to enjoy TCGs more, a technology has been proposed that uses an electronic device such as a smartphone to film the player playing the TCG, senses the cards placed, and supports the progress of the TCG based on the sensing results (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-154018 Summary of the Invention [Problem to be solved by the invention]

[0005] It is important to accurately obtain the game situation in order to support the progress of TCG.

[0006] The present technology has been developed in consideration of such problems, and aims to provide a program, an information processing method, and an information processing device that can acquire the game situation. [Means for solving the problem]

[0007] To solve the above-mentioned problem, the first technology is a program that causes a computer to execute an information processing method that detects the position of a card in an image captured by a camera of an area in which game cards are placed, cuts out the area of the card from the image based on the position of the card to generate a cutout image, and obtains the game situation based on the cutout image.

[0008] The second technology is an information processing method that detects the position of a card in an image captured by a camera of the area in which the game cards are placed, cuts out the area of the card from the image based on the position of the card to generate a cutout image, and obtains the game situation based on the cutout image.

[0009] The third technology is an information processing device that includes a card position detection unit that detects the position of a card in an image captured by a camera of the area in which game cards are placed, a cutout unit that cuts out the area of the card from the image based on the position of the card to generate a cutout image, and a situation acquisition unit that acquires the situation of the game based on the cutout image. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing the configuration of a TCG system 1000. FIG. [Figure 2] FIG. 10 is a diagram showing a card C used for TCG. [Figure 3] FIG. 10 is a diagram showing a card C used for TCG. [Figure 4] FIG. 10 is a diagram showing an information body P used for TCG. [Figure 5] 1 is a block diagram showing a hardware configuration of an information processing device 100. FIG. [Figure 6] FIG. 1 is a diagram illustrating processing blocks of an information processing device 100 according to a first embodiment. [Figure 7] 3 is a flowchart showing processing in the information processing device 100. [Figure 8] 10 is an explanatory diagram of a process for recognizing the mat 10. FIG. [Figure 9] FIG. 10 is an explanatory diagram of image keystone correction. [Figure 10] FIG. 10 is a diagram showing an image that has been subjected to keystone correction. [Figure 11] FIG. 10 is a diagram showing a state in which a card is placed on top of another card. [Figure 12] FIG. 10 is an explanatory diagram of generation of an output result. [Figure 13] FIG. 10 is an explanatory diagram of generation of an output result. [Figure 14] FIG. 10 is an explanatory diagram of a process using the output result. [Figure 15] FIG. 1 is an explanatory diagram of a TCG system 1000 for playing a TCG online. [Figure 16] FIG. 10 is a diagram illustrating processing blocks of an information processing device 100 according to a second embodiment. [Figure 17] 10 is a flowchart showing a process based on a motion detection result. [Figure 18] 10 is a flowchart showing a process based on a foreign object detection result. [Figure 19] FIG. 10 is a diagram showing a modified design of card C. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be made in the following order. First Embodiment [Configuration of TCG system 1000] [Configuration of information processing device 100] [Processing in information processing device 100] <Second embodiment> [Configuration of information processing device 100] [Processing in information processing device 100] <Modification>

[0012] First Embodiment [Configuration of TCG system 1000] The configuration of a TCG system 1000 will be described with reference to Fig. 1. The TCG system 1000 is made up of an information processing device 100 and a display device 200.

[0013] The present technology captures a TCG match between multiple players using the camera 106 of the information processing device 100, and performs processing based on the captured image.

[0014] In order for multiple players to play a TCG, a rectangular mat 10 is placed on a flat surface such as the floor or a table to specify the positions and areas where cards are placed. Two players, Player 1 and Player 2, face each other across the mat 10, and the TCG begins by each player placing cards on the mat 10 according to the specified rules.

[0015] The mat 10 is a square with the same length and width dimensions, for example, 60 cm x 60 cm. However, the size and shape of the mat 10 are not limited to a specific one, and the present technology can be implemented even without the mat 10.

[0016] The left half of the mat 10 is the area where player 1 places their cards, and the right half is the area where player 2 places their cards.

[0017] Cards C include, for example, cards that show information such as the appearance, name, skills, characteristics, and stamina of a character (which may be any person, animal, monster, robot, vehicle, building, etc.), as shown in Figure 2, and cards that give specific effects, influences, meanings, etc. (hereinafter referred to as effects, etc.) to other cards C, as shown in Figure 3. Note that the rules of the TCG may allow cards C that show characters to give effects, etc. to other cards C. One TCG may have approximately 5,000 types of cards C.

[0018] Note that the character is merely an example, and what card C shows is not limited to a character, but may also be a picture, a mark, a number, a letter, a pattern, or a combination thereof. Card C may show anything as long as the player can recognize that it is a different card.

[0019] Each card C has identification information attached thereto for identifying its type. The identification information may be a character string, a picture, a mark, a design, or a combination thereof, but may be anything that allows a player to recognize that the cards are different. The cards C may be colored differently depending on their type, or may all be the same color.

[0020] In the TCG of this embodiment, it is assumed that there is a rule that stacking multiple cards C can produce various effects.

[0021] The shape of the card C may be a square, a disk, a triangle, a polygonal shape with more sides, or a free shape. The material of the card C may be any material, such as paper, metal, plastic, or synthetic resin such as acrylic.

[0022] Furthermore, the TCG in this embodiment has an object (hereinafter referred to as information body P) that indicates information about the TCG, separate from the cards, as shown in Figure 4. In this embodiment, the information about the TCG indicated by the information body P is a numerical value indicating the effect, etc., given to card C. Since the information body P indicates the effect, etc., given to card C, one example of how to use the information body P is to place it on top of card C. Note that the information body P may be a numerical value indicating an effect, etc., or other information according to the TCG, such as a numerical value indicating an amount, score, points, etc.

[0023] In this embodiment, there are multiple types of information objects P with different numerical values, such as 100, 50, and 10. Each type of information object P may be colored differently, or all may be the same color. Examples of information objects P include cards, sheets, medals, stickers, and stones. The shape of the information objects P may be flat, disc-shaped, spherical, cubic, rectangular, or free-form. The material of the information objects P may be any material, such as paper, metal, plastic, or synthetic resin such as acrylic.

[0024] The information processing device 100 includes a camera 106 and is installed at a higher position than the mat 10 using a stand, tripod, stand, or the like so that the mat 10 can be photographed by the camera 106. In this embodiment, the information processing device 100 is a smartphone.

[0025] The display device 200 is a device capable of displaying images, such as a television, a display device, a personal computer, a tablet terminal, etc. The display device 200 is assumed to have a function of outputting sound from a built-in speaker or a connected external speaker.

[0026] The display device 200 is connected to the information processing device 100 by wired or wireless connection. Wired connection methods include, for example, HDMI (registered trademark) (High-Definition Multimedia Interface), USB (Universal Serial Bus), and MHL (Mobile High-Definition Link), and wireless connection methods include, for example, Wi-Fi and wireless LAN (Local Area Network). Any method may be used as long as it can connect the display device 200 and the information processing device 100.

[0027] 1, the information processing device 100 is placed on one side of the mat 10 and the display device 200 is placed on the other side, but the information processing device 100 and the display device 200 may be placed on the same side. In that case, it is preferable to flip the image displayed on the display device 200 left and right. This makes the state of the mat 10 in the real world match the state of the mat 10 displayed on the display device 200, making it easier for players to play the TCG while looking at the display on the display device 200.

[0028] A TCG application is installed in the information processing device 100. The TCG application has a function of displaying an image captured by the camera 106 of the information processing device 100 on the display unit 109 of the information processing device 100 in real time, and further outputting the image to the display device 200. As a result, the image of the mat 10 captured by the camera 106 is displayed on the display unit 109 of the information processing device 100 and the display device 200, and the player can view it.

[0029] [Configuration of information processing device 100] Next, the hardware configuration of the information processing device 100 will be described with reference to FIG.

[0030] The CPU 101 functions as an arithmetic processing unit that performs various processes, and controls the entire information processing device 100 and each unit. The CPU 101 executes various processes in accordance with programs stored in the ROM 102 or the nonvolatile memory unit 104, or programs loaded from the storage unit 111 to the RAM 103. The nonvolatile memory unit 104 may be, for example, an EEPROM (Electrically Erasable Programmable Read Only Memory). The RAM 103 also stores data and the like required for the CPU 101 to execute various processes, as appropriate.

[0031] The CPU 101, ROM 102, RAM 103, and nonvolatile memory unit 104 are interconnected via a bus, to which an input / output interface 105 is also connected.

[0032] To the input / output interface 105, a camera 106, an IMU (Inertial Measurement Unit) 107, a distance sensor 108, a display unit 109, an input unit 110, a storage unit 111, a communication unit 112, and a drive 113 are connected.

[0033] The camera 106 includes an imaging element, a signal processing circuit, etc., and captures RGB (Red, Green, Blue) or monochrome video or images. The imaging element may be a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc.

[0034] The camera 106 may be a digital still camera, an action camera, or the like, which is separate from the information processing device 100, or a standalone camera that is connected to the information processing device 100 by wire or wirelessly.

[0035] In TCGs, card C is sometimes placed in a transparent protective film called a sleeve. However, placing card C in a sleeve reflects light, preventing part or all of card C from being displayed in the image, which may reduce the accuracy of card C identification by the information processing device 100. In addition, the card C itself, rather than the sleeve, may reflect light. Furthermore, some cards C have a hologram or other surface treatment that makes them shiny. These factors may also reduce the accuracy of card C identification by the information processing device 100. Therefore, by attaching a polarizing filter to the lens of the camera 106, it is possible to remove the reflected light component and prevent a decrease in the accuracy of card C identification by the information processing device 100. Regarding the rainbow-colored glow treatment, it is possible to reduce the reflected component by augmenting the learning data.

[0036] To solve the problem of reflections, a polarization image sensor can be used in the camera 106. A polarization image sensor is an image sensor equipped with a four-directional polarizer. A polarization image sensor has four polarizers with polarization angles of 0°, 45°, 90°, and 135° per pixel, allowing it to capture four polarization images in a single capture. This eliminates the need to capture images with different polarization filters, as is the case with polarizing filters, and it is advantageous in capturing moving objects. By using a polarization image sensor, it is possible to reduce or eliminate reflected light from the sleeve or the card C itself. Capturing four polarization images using a polarization filter requires repeating the capture four times with each polarizing filter, which can be disappointing or disappointing to players. However, using a polarization image sensor allows it to capture four polarization images in a single capture, ensuring real-time performance and preventing players from losing interest or losing interest.

[0037] The IMU 107 acquires the movement and attitude of the information processing device 100 using a two-axis or three-axis acceleration sensor, an angular velocity sensor, a gyro sensor, and the like.

[0038] The distance sensor 108 measures the distance to the subject and outputs depth information as sensing data. The distance sensor may be a Time of Flight (ToF), Light Detection and Ranging (LiDAR), infrared sensor, ultrasonic sensor, color stereo camera, IR (Infrared) stereo camera, monocular camera, or the like. The distance sensor 108 may also be a triangulation sensor using one IR camera and structured light. The distance sensor 108 may also be a combination of multiple sensors.

[0039] The display unit 109 is a display device such as a liquid crystal display or an organic EL (Electroluminescence) display provided in the information processing device 100, or a separate display device connected to the information processing device 100, or the like.

[0040] The input unit 110 is, for example, a variety of controls or operation devices such as a touch panel or a touch pad. The input unit 110 may also be a keyboard, a mouse, keys, a dial, a remote controller, etc. The input unit 110 detects the player's operation, and the CPU 101 interprets the signal corresponding to the input operation.

[0041] The storage unit 111 is a large-capacity storage medium such as a hard disk, a flash memory, etc. The storage unit 111 stores various applications, data, information, etc.

[0042] The communication unit 112 is a communication terminal or a communication module for various purposes, which performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, or the like.

[0043] A drive 113 is connected to the input / output interface 105 as needed. A removable storage medium 114 is appropriately attached via the drive 113. The removable storage medium 114 includes a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like.

[0044] The drive 113 can read data files such as programs used in each process from the removable storage medium 114. The read data files are stored in the storage unit 111. In addition, the computer programs read from the removable storage medium 114 are installed in the storage unit 111 as needed.

[0045] In the information processing device 100, for example, a program or application for processing of the present technology can be installed via network communication by the communication unit 112 or a removable storage medium 114. The program or application may also be stored in advance in the ROM 102, the storage unit 111, or the like. Images captured by the camera 106 or processing results of AI image processing may also be received, and the images or processing results may be stored in the storage unit 111 or the removable storage medium 114.

[0046] Next, with reference to FIG. 6, the processing blocks of the information processing device 100 in the first embodiment will be described.

[0047] The image processing unit 151 cuts out an image of the area of the mat 10 from the frame images constituting the video captured by the camera 106, and performs keystone correction processing on the image of the mat 10. In this technology, each successive frame image constituting the video captured by the camera 106 is the unit of processing.

[0048] The card position detection unit 152 detects the position of the card from the image of the mat 10. The card position detection unit 152 can also detect the number of cards placed on the mat 10 by detecting the position of the card.

[0049] The cutout unit 153 cuts out the area of the card whose position has been detected by the card position detection unit 152 from the frame image to generate a cutout image.

[0050] The card identifying unit 154 identifies the type of card C placed on the mat 10 based on the cut-out image. The type and number of cards C are the game situations in the claims. The card identifying unit 154 includes a first card identifying unit 155 and a second card identifying unit 156.

[0051] The first card identification unit 155 performs a process of identifying the type of card C for all clipped images.

[0052] The second card identification unit 156 performs a process to identify the type of card C for a cutout image that could not be identified by the first card identification unit 155. The first card identification unit 155 is unable to identify a card C when another card C is placed on top of it and part of the card C is hidden. Therefore, the second card identification unit 156 identifies the type of card C based on the identification information on the exposed part of the surface of the card C, rather than on the entire surface of the card C.

[0053] The output generation unit 157 generates an output result for the latest frame image based on the identification result of the type of card C in the latest frame image and the output result for a past frame image (the frame image immediately preceding the current frame image). The output generation unit 157 may hold the output result for the immediately previous frame image, or may store the output result for the past frame image in the storage unit 111 and read out the output result for the immediately previous frame image from the storage unit 111 when performing processing.

[0054] The object detection unit 158 detects information bodies P present on the cards C whose types have been identified by the first card identification unit 155. The types and number of information bodies P are also game situations within the scope of the claims.

[0055] The sum calculation unit 159 sums up the numerical values indicated by the plurality of information objects P detected by the object detection unit 158 in the same clipped image.

[0056] The card identification unit 154 and the object detection unit 158 correspond to the status acquisition unit that acquires the status of the TCG in the claims.

[0057] The information processing device 100 is configured as described above. In this embodiment, the information processing device 100 is configured by a smartphone. However, the information processing device 100 may also be configured by an electronic device such as a personal computer or a tablet terminal. The processing blocks of the information processing device 100 and the information processing method are realized by the smartphone or such electronic device executing a program according to the present technology. The program may be installed in advance on the smartphone or such electronic device, or may be distributed by download or storage medium, etc., and installed by a player, etc. Furthermore, the processing blocks of the information processing device 100 may be configured on a cloud server.

[0058] A cloud server is not limited to being configured by a single computer device, but may be configured by a system of multiple computer devices. The multiple computer devices are systemized, for example, by a LAN (Local Area Network) or the like. Also, multiple computer devices located in remote locations may be systemized by a VPN (Virtual Private Network) using the Internet or the like. The multiple computer devices may include computer devices as a server group (cloud) available through a cloud computing service.

[0059] [Processing in information processing device 100] The following describes the processing in the information processing device 100. First, as a premise, it is assumed that a mat 10 indicating an area on which a card C is placed is placed on a flat surface such as a floor, as shown in Fig. 1, and that the information processing device 100 is set up so that the mat 10 is photographed by a camera 106.

[0060] The information processing device 100 recognizes the mat 10 from the video captured by the camera 106. Recognition of the mat 10 can be performed by a DNN (Deep Neural Network). It can also be performed by machine learning, artificial intelligence, etc., such as a CNN (Convolutional Neural Network) or an RF (Random Forest). For example, a neural network or deep learning is used as a learning method for machine learning. It is also possible to use known object recognition techniques, such as a template matching method or a matching method based on brightness distribution information of a subject.

[0061] When the information processing device 100 recognizes the mat 10, it displays a trapezoidal frame F on the display unit 109 to identify the area in the image corresponding to the mat 10 based on the recognition result, as shown in Fig. 8. This frame F is used to identify the area (sometimes referred to as the target area) from which the information processing device 100 acquires the status of the TCG. It is desirable to install the information processing device 100 so that the front side (bottom side) of the frame F is aligned with the lower edge of the display unit 109 (the lower edge of the image to be displayed). Note that the image captured by the camera 106 of the information processing device 100 is also displayed on the display device 200 in real time, and therefore the frame F is also displayed on the display device 200.

[0062] The mat 10 is a commercially available product, and its size and shape are known. Therefore, based on the distance between the camera 106 and the plane on which the mat 10 is installed, the tilt angle of the camera 106 with respect to the plane, and the FoV (Field of View) of the camera 106, it is possible to identify the shape and size of the trapezoid that the mat 10 will appear as on the display unit 109. The information processing device 100 draws and displays a frame F based on the identification result. The distance between the camera 106 and the plane can be acquired by the distance sensor 108. Furthermore, the tilt angle of the camera 106 with respect to the plane can be acquired by the IMU 107.

[0063] Alternatively, a method may be used in which markers are placed at the four corners of the mat 10, and the markers are detected from the image captured by the camera 106 to display the frame F. Alternatively, the corners of the mat 10 may be detected from the image captured by the camera 106, and the frame F may be displayed based on the corners. Alternatively, the frame F may be displayed based on the position estimation results and map creation results obtained by SLAM (Simultaneous Localization and Mapping).

[0064] It should be noted that the positional relationship between the target area and mat 10 can be fixed in an optimal state by installing a TCG-dedicated camera device with a fixed angle of view at a specific tilt angle on a TCG-dedicated camera fixing stand at a specific position relative to mat 10. In this case, the above-described process using frame F is not necessary.

[0065] The function of recognizing the mat 10 may be provided in the information processing device 100 or in the TCG application.

[0066] This technology can also be used when the mat 10 is not used and the card C is placed directly on the floor.

[0067] The flowchart in Fig. 7 will be described. First, in step S11, the image processing unit 151 cuts out the area of the mat 10 indicated by the frame F from the frame image constituting the video captured by the camera 106, and performs keystone correction processing on the cut-out image of the mat 10.

[0068] As shown in Fig. 9, the keystone correction process can be performed based on the attitude of the camera 106 obtained by the IMU 107, the distance (depth) to the plane on which the mat 10 is placed obtained by the distance sensor 108, and the FoV (field of view) of the camera 106. As a result of performing the keystone correction process, the trapezoidal image of the mat 10 is converted into a rectangular image as if it were taken from directly above, as shown in Fig. 10. Note that the keystone correction process can also be performed based only on the distance to the plane obtained by the distance sensor 108 and the FoV of the camera 106.

[0069] As described above, in a TCG, the position and area where the card C is placed are determined by the mat 10. Therefore, the information processing device 100 does not need to acquire the TCG situation (such as identifying the type of card C) for the entire frame image captured by the camera 106, but only needs to acquire the situation for the target area defined by the mat 10.

[0070] Next, in step S12, the card position detection unit 152 detects the positions of the cards C from the image of the mat 10 that has been subjected to trapezoidal correction. The card position detection unit 152 detects the positions and number of cards C present on the mat 10 by detecting the corners of the cards C. The corner detection of the cards C can be performed using, for example, the Harris corner detection method or a feature point detection method. It can also be performed using machine learning, artificial intelligence, etc., such as DNN, CNN, and RF. The card position detection unit 152 supplies the detected positions and number of cards C to the cutout unit 153 as detection results.

[0071] Next, in step S13, the cutout unit 153 cuts out the area of card C from the frame image based on the position of card C detected by the card position detection unit 152 to generate a cutout image. The cutout unit 153 also performs trapezoid correction processing on the cutout image. The cutout unit 153 supplies the cutout image to the card identification unit 154. The number of cutout images is the same as the number of cards C detected by the card position detection unit 152.

[0072] Next, in step S14, the card identification unit 154 performs a process of identifying the type of card C for the cutout images. The card identification unit 154 first identifies the type of card C using the first card identification unit 155. The first card identification unit 155 performs a process of identifying the type of card C for all cutout images. The first card identification unit 155 identifies the type of card C based on the features of the entire surface of the card C.

[0073] The first card identification unit 155 can identify the type of card C using a DNN. Unlike conventional methods such as feature point matching, identifying the type of card C using a DNN makes it possible to identify thousands of different types of card C almost in real time. The first card identification unit 155 may identify the type of card C using machine learning such as CNN or RF, artificial intelligence, or the like.

[0074] The first card identification unit 155 needs to learn in advance about all cards C to be identified. For example, if there are 5,000 types of cards C in a TCG, the first card identification unit 155 needs to learn in advance about those 5,000 types of cards C. It is desirable to perform the learning using both actual photographs of the cards C and CG (Computer Graphics) images representing the designs of the cards C. This can improve the accuracy of identification.

[0075] The second card identification unit 156 performs a process of identifying the type of card C for a cutout image for which the first card identification unit 155 was unable to identify the type of card C. The first card identification unit 155 is unable to identify the type of card C when another card C is placed on top of the card C, hiding part of the underlying card C. Therefore, the second card identification unit 156 identifies the type of the underlying card C based on the characteristics of the exposed part that is not hidden by the other card C placed on top, rather than on the entire surface of the card C.

[0076] The second card identification unit 156 can use DNN to identify the type of card C. The second card identification unit 156 may also use machine learning such as CNN or RF, artificial intelligence, or the like to identify the type of card C.

[0077] For example, in a TCG, when multiple cards C are stacked, if only a specific card C is placed under the other cards C and only a portion of the card C is exposed, the second card identification unit 156 does not need to learn about all cards C, but only needs to learn in advance about the specific card C that is placed under the other cards C. For example, if there are 5,000 types of cards C in total, and of those, there are four specific types of cards C that are placed under the other cards C, the second card identification unit 156 only needs to learn in advance about those four specific types of cards C. Therefore, since there are only a small number of types of cards C to be identified by the second card identification unit 156, erroneous identification can be reduced. However, if there is a possibility that all cards C will be placed under other cards C according to the rules of the TCG, the second card identification unit 156 needs to learn about all cards C.

[0078] The second card identification unit 156 identifies the type of card C based on the identification information present not on the entire surface of card C, but on the exposed area that is not hidden by other cards C stacked on top of it, as shown in Figure 11.

[0079] In a TCG, if there is a rule that "when stacking cards, place the top card C so that the top part of the bottom card C is exposed" or if there is such a custom among players, it is preferable that the identification information be prominently displayed on the top part of the bottom card C, as shown in Figures 3 and 11. This can improve the accuracy of identifying the type of card C by the second card identification unit 156. Since this varies depending on how the cards C are stacked, for example, in a TCG, if there is a rule that "when stacking cards, place the top card C so that the bottom part of the bottom card C is exposed" or if there is such a custom among players, it is preferable that the identification information be prominently displayed on the bottom part of the card C.

[0080] 2, the second card identification unit 156 can identify the type of the underlying card C based on the identification information (a character string indicating the character's name in the example of FIG. 2). Also, if a picture representing the character is exposed, the second card identification unit 156 can also identify the type of card C based on the picture.

[0081] Furthermore, if different colors are assigned to different types of cards C, color detection can be used to identify the type of card C. The type of card C may also be identified based on both the identification information and the color.

[0082] Therefore, the second card identification unit 156 can identify the type of the underlying card C even when a card C is placed on top of it as shown in Fig. 11A. The cards C may be stacked horizontally as shown in Fig. 11B, or diagonally as shown in Fig. 11C. Furthermore, three or more cards C may be stacked as shown in Fig. 11D. In this case, each of the multiple cards with a card C placed on top can be identified.

[0083] It is desirable that at least the top third of card C, which is placed on the bottom when stacked, is exposed, but this is just one example, and by improving the identification accuracy using DNN, etc., it is possible to identify the type of card C even with a smaller exposed area.

[0084] The card identification unit 154 outputs the identification results from the first card identification unit 155 and the second card identification unit 156 to the output generation unit 157. In addition, the card identification unit 154 supplies the cut-out image in which the type of card C has been identified by the first card identification unit 155 to the object detection unit 158.

[0085] The first card identifying unit 155 and the second card identifying unit 156 identify the type of card C for each cut-out image, so even if there are multiple copies of the same card C on the mat 10, they can each be identified.

[0086] Some TCGs release new types of cards on a regular or irregular basis. When new cards are released, the new cards can be identified by learning about them in the card identification unit 154. Furthermore, if a player plays a TCG other than the TCG in this embodiment, the card identification unit 154 must learn about the cards of that TCG in advance.

[0087] Next, in step S15, the output generation unit 157 combines the identification result of card C in the latest frame image with the output result in the past frame image (the previous frame image) to generate an output result indicating the type and number of cards C in the latest frame image.

[0088] 12 and 13, the generation of output results by the output generation unit 157 will be described. As shown in Fig. 12A, for example, assume that the identification results by the card identification unit 154 in frame (n) are card C1, card C2, and card C3, and the output results by the output generation unit 157 in the previous frame (n-1) are card C1 and card C2.

[0089] In this case, card C3, which did not exist in frame (n-1), is newly identified in frame (n), and therefore the likelihood of card C3 appears in the synthesis result by the output generation unit 157 (the output result of frame (n)). Here, as an example, the range of change in likelihood in one frame is assumed to be ±20. If the likelihood exceeds a predetermined threshold, it is determined that a card is present, and the type of that card is included in the output result. As an example, the threshold is assumed to be 50. Therefore, in the output result of frame (n), card C3 is not yet present and has not been identified.

[0090] Next, as shown in FIG. 12B, it is assumed that the output results of frame (n) are card C1, card C2, and card C3 (likelihood 20), and the identification results of the next frame (n+1) are card C1, card C2, and card C3.

[0091] In this case, card C3 was identified in frame (n+1), so the likelihood of card C3 in the synthesis result (output result of frame (n+1)) increases to 40. At this point, the likelihood does not exceed the threshold, so card C3 is not yet present and has not been identified in the output result of frame (n+1).

[0092] Next, as shown in FIG. 12C, the output results for frame (n+1) are cards C1, C2, and C3 (likelihood 40), and the identification results for the next frame (n+2) are cards C1, C2, and C3.

[0093] In this case, card C3 was identified in frame (n+2), so the likelihood of card C3 in the synthesis result (output result of frame (n+2)) increases to 60. Because the likelihood of card C3 exceeds the threshold, card C3 is considered to exist and has been identified, and the output result of frame (n+2) is card C1, card C2, and card C3.

[0094] On the other hand, as shown in Figure 13A, if the output results of frame (n) are card C1, card C2, and card C3 (likelihood 20), the identification results of the next frame (n+1) are card C1 and card C2, and card C3 is not identified.

[0095] In this case, in the output result of frame (n+1), cards C1 and C2 are present, and the likelihood of card C3 drops from 20 to 0. Because the likelihood is below the threshold, card C3 is deemed not to be present in the output result of frame (n+1).

[0096] Also, as shown in Figure 13B, if the output results of frame (n+2) are card C1, card C2, and card C3 (likelihood 60), the identification results of the next frame (n+3) are card C1 and card C2, and card C3 is not identified.

[0097] In this case, in the output result of frame (n+3), cards C1 and C2 are present, and the likelihood of card C3 drops from 60 to 40. As a result, the likelihood of card C3 falls below the threshold, so card C3 is not present in the output result of frame (n+3) and has not been identified.

[0098] In this way, whether the type of card C is identified or not identified by the card identification unit 154, the single identification result is not output as is, but the likelihood is increased or decreased depending on the identification result, and the likelihood is compared with a threshold value to determine the output result. Since the identification result may include erroneous detection or erroneous recognition, this process can make the output result highly reliable.

[0099] Then, as the output result for the latest frame image, the type and number of the identified cards C are output to the outside. The output result is output to, for example, a TCG application or the like.

[0100] Returning to the description of the flowchart in Fig. 7, next, in step S16, the object detection unit 158 detects an information body P from the cutout image in which the type of card C has been identified by the first card identification unit 155. The detection of the information body P can be performed by DNN. It can also be performed by machine learning such as CNN or RF, artificial intelligence, or the like.

[0101] The object detection unit 158 needs to learn in advance about all information objects P to be detected. Since the information object P is placed on top of the card C when used, it exists within a cut-out image cut out as an area of the card C. Therefore, the detection of the information object P is also performed on the cut-out image. If the information object P is assigned a different color for each numerical value, the position and type of the information object P can also be identified by color detection. Depending on the progress of the TCG, there may be cases where the information object P has not yet been used at the processing stage. The object detection unit 158 outputs the detection result to the sum calculation unit 159.

[0102] Next, in step S17, the sum calculation unit 159 sums up the numerical values indicated by the multiple information objects P detected by the object detection unit 158 in the same clipped image. For example, if there are two information objects P indicating the numerical value "100", one information object P indicating the numerical value "50", and one information object P indicating the numerical value "10" on one clipped image in which the type of card C has been identified, the sum of the numerical values is calculated to be 260. This makes it possible to output as information the sum of the numerical values indicating the effect, etc., given to the card C on which the information objects P are placed.

[0103] Note that step S15, step S16 and step S17 may be performed in the reverse order, or may be performed simultaneously or almost simultaneously.

[0104] The processing in the first embodiment is carried out as described above. According to the first embodiment, it is possible to identify the type of card C placed on the mat 10. It is also possible to detect the number of cards C. It is also possible to detect the information object P used in the TCG. These enable the status of the TCG to be acquired.

[0105] The identification result of card C can be used for various processes by outputting it to an external device such as a TCG application. For example, the identification result of card C can be used to display a TCG interface such as that shown in Fig. 14 on the display device 200.

[0106] The TCG interface consists of a video display area VD in the center, a first card display area D1 on the left, and a second card display area D2 on the right.

[0107] The video display area VD displays in real time the video captured by the camera 106 of the information processing device 100. The first card display area D1 displays a display card image CP corresponding to a card C placed in the left half of the mat 10. The second card display area D2 displays a display card image CP corresponding to a card C placed in the right half of the mat 10. Therefore, in the example of FIG. 14, the first card display area D1 displays a display card image CP corresponding to the card C placed on the mat 10 by player 1. Furthermore, the second card display area D2 displays a display card image CP corresponding to the card C placed on the mat 10 by player 2.

[0108] The display card image CP is image data created using CG that shows the same design as the actual card C. In order to display the display card image CP in the first card display area D1 and the second card display area D2, the display card image CP is associated in advance with the type of the actual card C. Display card images CP are prepared in advance for all cards C used in the TCG and stored in the storage unit 111 or the like.

[0109] Then, based on the identification result of the card C, the display card image CP corresponding to the card C placed on the mat 10 is read out and displayed in the first card display area D1 and the second card display area D2.

[0110] 14, the first card display area D1 and the second card display area D2 can both display eight display card images CP, but it is also possible to display more display card images CP. For example, it is possible to display more display card images CP by scrolling up and down or left and right, or if there are more than eight display card images CP, the display size of the display card images CP may be reduced so that more display card images CP can be displayed.

[0111] Since this technology can also detect the position of card C, the display card image CP corresponding to the card C placed in the left half of the mat 10 is displayed in the first card display area D1. Also, the display card image CP corresponding to the card C placed in the right half of the mat 10 is displayed in the second card display area D2. This allows the player to check the status of the TCG on the large screen of the display device 200.

[0112] As shown in Fig. 14, the display card image CP can also be displayed in the video display area VD. For example, it is possible to display the display card image CP corresponding to the card C placed on the mat 10, and to produce a TCG effect by moving or rotating the display card image CP.

[0113] Furthermore, by previously associating the type of card C with detailed information about card C in the TCG application, the detailed information about card C can be displayed on the TCG interface according to the identification result of card C. This is convenient when, for example, the opposing player plays a card C that the player does not recognize. The detailed information about card C may be displayed in a position corresponding to card C displayed in the video display area VD, or in a position corresponding to the display card image CP displayed in the first card display area D1 or the second card display area D2. The detailed information about card C includes, for example, the effect and status of card C.

[0114] Furthermore, by previously associating sounds with types of cards C in the TCG application, it is possible to output specific sounds from the display device 200 according to the types of cards C. Examples of sounds include background music (BGM), sound effects, audio effects, audio guides, and automatic commentary. For example, sounds may be output in accordance with the timing when a player places a card C or when a card C is turned over. This allows the TCG to be enhanced.

[0115] Furthermore, by previously associating the type of card C with an image in the TCG application, it is possible to display a specific image in the TCG interface according to the type of card C. Examples of images include CG effects, movies, etc. For example, an image may be displayed in accordance with the timing when a player places a card C or turns over a card C. This allows the TCG to be enhanced.

[0116] This technology is also useful when playing a TCG online using the Internet. For example, as shown in FIG. 15, consider a case where player 1 and player 2, who are in different locations, are playing a TCG online. Player 1 places a mat 10A and prepares an environment for playing the TCG using an information processing device 100A and a display device 200A. Player 2 places a mat 10B and prepares an environment for playing the TCG using an information processing device 100B and a display device 200B. The information processing device 100A and the information processing device 100B are connected via the Internet.

[0117] It is assumed that information processing device 100A and information processing device 100B are equipped with the functionality of an online battle application. The online battle application may be pre-installed on information processing device 100A and information processing device 100B, or may be downloaded, distributed on a storage medium, or the like, and installed by the players.

[0118] The online battle application has a communication processing function that uses the communication unit 112 of the information processing device 100. This allows the information processing device 100A and the information processing device 100B to send and receive information such as images captured by the camera 106 and the identification result of the card C.

[0119] In addition, the online battle application has the function of synthesizing an image of the mat 10A captured by the camera 106 of the information processing device 100A with an image of the mat 10B captured by the camera 106 of the information processing device 100B to generate an image that makes it appear as if player 1 and player 2 are in the same space.

[0120] The information processing device 100A on the player 1 side combines the image captured by its own camera 106 with the image of the mat 10B on the player 2 side captured by the camera 106 of the information processing device 100B, and displays the combined image on the display device 200A. Furthermore, the information processing device 100B on the player 2 side combines the image captured by its own camera 106 with the image of the mat 10 on the player 1 side captured by the camera 106 of the information processing device 100A, and displays the combined image on the display device 200B.

[0121] In such a case, if the image quality of the image captured by the camera 106 of the information processing device 100A is poor or if the letters on the card C placed on the mat 10A are small, the card C on the mat 10A displayed on the display device 200B is difficult to see, making it difficult for player 2 to understand the situation of the game.

[0122] In addition, in TCGs, players may check information (sometimes called text) about their opponent's card C, but in online matches, even if they photograph the card with camera 106, it is difficult to visually check the information about card C, so there is also the problem that players must look up the information about card C themselves.

[0123] Using the present technology, player 1's card C can be identified by information processing device 100A and the identification result can be transmitted to player 2's information processing device 100B, thereby displaying information about player 1's card C on display device 200B. Conversely, player 2's card C can be identified by information processing device 100B and the identification result can be transmitted to player 1's information processing device 100A, thereby displaying information about player 2's card C on display device 200A. This allows players to easily check information about their opponent's card C even in an online TCG.

[0124] The online battle application may run on a server, and information processing device 100A and information processing device 100B may be connected to the server via the Internet.

[0125] Furthermore, when streaming a video of someone playing a TCG via a video streaming service or the like, it is possible to have a virtual celebrity such as a VTuber play the TCG and stream the footage by creating a video for streaming using the output results of the information processing device 100, rather than using real-world footage captured by the camera 106. When streaming using real-world footage captured by the camera 106, there is a risk that a player's face may accidentally appear in the video, but this method eliminates such a risk.

[0126] In addition, by combining this technology with a display device (such as a wearable eyeglasses-type device or a head-mounted display) worn by a player for AR (Augmented Reality) or MR (Mixed Reality) equipped with a camera, it is possible to realize a TCG application that uses AR or MR.

[0127] In addition, the AR or MR display terminal can be used together with the above-mentioned online TCG application. Also, if the relative position of the player with respect to the mat 10 is known, it is possible to place a virtual opponent on the opposite side of the player across the mat 10.

[0128] A display terminal worn by a player, such as a glasses-type wearable device or a head-mounted display, may be provided with the functions of the information processing device 100 in advance.

[0129] This technology can enhance the entertainment value of games while retaining the benefits of analog games that use real objects such as cards, and can provide players with a new gaming experience.

[0130] <Second embodiment> [Configuration of information processing device 100] Next, a second embodiment of the present technology will be described. First, a processing block of an information processing device 100 in the second embodiment will be described with reference to Fig. 16. Note that the configurations of the TCG and the TCG system 1000 are the same as those in the first embodiment.

[0131] The information processing device 100 in the second embodiment differs from the first embodiment in that it includes a movement determination unit 160, a foreign object determination unit 161, and a hand detection unit 162.

[0132] The movement determination unit 160 determines whether the information processing device 100 is moving based on the sensing data supplied from the IMU 107. The movement determination unit 160 determines that the information processing device 100 is moving, for example, when an acceleration value as sensing data from the IMU 107 exceeds a predetermined threshold. The information processing device 100 is moving when, for example, a player is holding the information processing device 100 in their hand or when a stand or the like supporting the information processing device 100 is moving.

[0133] The foreign object determination unit 161 generates a three-dimensional point cloud of the foreign object from the depth information supplied from the distance sensor 108, and determines whether the three-dimensional point cloud is present in the space between the mat 10 and the camera 106 (a predetermined distance (e.g., several centimeters above the mat 10)), thereby determining whether a foreign object is present. The foreign object may be any object, such as a part of the player's body, such as a hand, or a box, cup, or the like.

[0134] In addition, it may be determined that a foreign object is present when a three-dimensional point cloud of the foreign object is detected in the space between the mat 10 and the camera 106, or it may be determined that a foreign object is present when the amount of the three-dimensional point cloud detected in that space is greater than or equal to a predetermined threshold.

[0135] In this way, foreign objects are detected three-dimensionally using depth information, so that foreign objects can be detected in the space between the mat 10 and the camera 106. Furthermore, by using depth information, various objects can be detected as foreign objects without being limited to a specific type of object.

[0136] The hand detection unit 162 detects information about the hands, such as the positions, shapes, and number of hands present in the space between the mat 10 and the camera 106, based on either or both of the depth information and the frame images constituting the video captured by the camera 106. The detection of information about the hands can be performed by a method using machine learning or deep learning, a method using template matching, a matching method based on brightness distribution information of the subject, a method using artificial intelligence, or the like.

[0137] The information processing device 100 of the second embodiment is configured as described above. The other processing blocks and hardware configuration of the information processing device 100 are the same as those of the first embodiment.

[0138] [Processing in information processing device 100] Next, a description will be given of the processing performed by the information processing device 100 of the second embodiment. Note that the processing for identifying the type of card C and the processing for detecting information objects P are the same as those in the first embodiment.

[0139] First, the process based on the determination result of the motion determination unit 160 will be described with reference to FIG.

[0140] In step S21, the information processing device 100 executes a process for identifying the type of card C.

[0141] Next, in step S22, the movement determination unit 160 determines whether or not the information processing device 100 is moving. If the information processing device 100 is not moving, the process proceeds to step S21 (No in step S22), and the card identification unit 154 continues the identification process for card C. In this way, the card identification unit 154 continues the identification process unless the movement determination unit 160 determines that the information processing device 100 is moving.

[0142] On the other hand, if the movement determining unit 160 determines that the information processing device 100 is moving, it notifies the card identifying unit 154 to that effect, and the process proceeds to step S23 (Yes in step S22).

[0143] Next, in step S23, the card identifying unit 154 stops the identification process for card C.

[0144] Next, in step S24, if the movement determination unit 160 determines that the information processing device 100 is moving, it notifies the card identification unit 154 of this fact, and the process proceeds to step S23 (Yes in step S24). Then, in step S23, the card identification unit 154 continues to suspend the identification process of card C.

[0145] On the other hand, if the movement determination unit 160 determines that the information processing device 100 is not moving, it notifies the card identification unit 154 of this fact, and the process proceeds to step S21 (No in step S24). Then, in step S21, the card identification unit 154 executes the identification process for card C, that is, it resumes the identification process for card C that had been stopped.

[0146] As described above, in the second embodiment, the identification process for card C is performed when the information processing device 100 is not moving, and the identification process for card C is stopped when the information processing device 100 is moving. This allows the identification process for card C to be performed only when the information processing device 100 is not moving and can capture a clear image, thereby improving the identification accuracy.

[0147] Next, processing based on the determination result of the foreign matter determining unit 161 will be described with reference to FIG.

[0148] First, in step S31, the information processing device 100 executes a process for identifying the type of card C.

[0149] Next, in step S32, the foreign matter determination unit 161 determines whether or not a foreign matter is present in the space between the mat 10 and the camera 106. If no foreign matter is present, the process proceeds to step 31 (No in step S32), and the card identification unit 154 continues the identification process for the card C. In this way, the card identification unit 154 continues the identification process unless the foreign matter determination unit 161 determines that a foreign matter is present.

[0150] On the other hand, if the foreign matter determining unit 161 determines that a foreign matter is present, it notifies the card identifying unit 154 to that effect, and the process proceeds to step S33 (Yes in step S32).

[0151] Next, in step S33, the card identifying unit 154 stops the identification process for card C.

[0152] Next, in step S34, if the hand detection unit 162 detects a hand in the space between the mat 10 and the camera 106, the process proceeds to step S35 (Yes in step S34).

[0153] Then, in step S35, the hand detection unit 162 outputs the detection information such as the hand position, hand shape, and number of hands to the outside.

[0154] On the other hand, if the hand detection unit 162 does not detect a hand in the space between the mat 10 and the camera 106 in step S34, the process proceeds to step S36 (No in step S34).

[0155] Next, in step S36, the foreign matter determination unit 161 determines whether or not a foreign matter is present in the space between the mat 10 and the camera 106. If a foreign matter is present, the process proceeds to step S33 (Yes in step S36), and the card identification unit 154 continues to stop the identification process.

[0156] On the other hand, if the foreign matter determination unit 161 determines that no foreign matter is present, it notifies the card identification unit 154 of this fact, and the process proceeds to step S31 (No in step S36). Then, in step S31, the card identification unit 154 executes the card identification process, that is, it resumes the card identification process that was stopped.

[0157] In this way, in the second embodiment, the card C is identified when there is no foreign object in the space between the mat 10 and the camera 106, and the identification process is stopped when there is a foreign object. This prevents the camera 106 from capturing an image of the foreign object, which could reduce the identification accuracy of the card C or result in an incorrect identification.

[0158] Furthermore, if a foreign object is present, the hand detection unit 162 performs processing to detect the hand. The TCG application can execute various processes using the detection information output from the hand detection unit 162. For example, by using the player's pointing finger as a UI (User Interface), the TCG application can progress through the TCG or execute specific processes.

[0159] Furthermore, when a player's hand is extended with a specific finger (for example, an index finger) and that finger is pointing at a card C that can be identified by position information, the TCG application can execute the effect, etc. indicated by that card C on the TCG and reflect it in the progress of the game. Also, it may be possible to use that finger to specify another card C on which the effect, etc. of card C will be applied.

[0160] Furthermore, when a player's hand is extended with a specific finger and that finger is pointing at a card C that can be identified by position information, the TCG application can display an image or output sound according to the effect, etc., indicated by the card C pointed at by that finger.

[0161] Also, options relating to the card C pointed at by the player's finger are displayed on the display device 200 so that the player can make a selection. The player's selection of the option may also be made by detecting the finger.

[0162] Also, specific hand or finger movements or shapes may be used to issue commands to the TCG application. Examples of commands include ending one's turn or surrendering (determining the winner of the game). A turn is a unit of game progress in a game involving multiple players, and each player takes their turn in a specific order.

[0163] <Modification> Although the embodiments of the present technology have been specifically described above, the present technology is not limited to the above-described embodiments, and various modifications based on the technical concept of the present technology are possible.

[0164] In the embodiment, it has been explained that card C indicates information such as the appearance, name, skills, characteristics, and physical strength of a character (person, animal, monster, robot, vehicle, etc.), or indicates the parameters necessary for the character indicated by card C to use skills, etc. However, card C may also be a card representing other elements, such as equipment, items, etc., or any card that indicates information related to the TCG.

[0165] In this embodiment, as an example, the card stacked underneath card C is a card that gives a specific effect to other card C, but the card C stacked underneath is not limited to this and may be a card that indicates other information, or any other card.

[0166] Alternatively, a card C showing a character may be placed on top of another card C showing a character. In this case, the second card identification unit 156 identifies the type of card based on the characteristics of the exposed part of the card C placed underneath. To do this, the second card identification unit 156 needs to learn about all the cards C showing characters in advance.

[0167] In the embodiment, the rule that a specific effect can be given to card C by stacking card C has been described, but the present technology can also be used with other rules. For example, placing a card indicating a change under a character card C can change the character, or placing a card indicating an item under a character card can use the item on the character, etc.

[0168] The design of card C shown in Figure 3 is merely an example, and any design is acceptable as long as the identification information is depicted as large as possible so that it is easily recognizable. For example, the identification information may be larger than in Figure 3, as shown in Figures 19A and 19B, or the identification information may be depicted in the upper center of card C, as shown in Figure 19C.

[0169] The information about the TCG indicated by the information body P may be any information used in the TCG, such as numbers indicating the effects given to card C, numbers, letters, pictures, marks, etc. indicating the state of card C, other than numbers. Furthermore, the information body P may indicate the effects given to card C using any one or combination of letters, pictures, marks, colors, and patterns other than numbers.

[0170] The TCG may not use the information object P. If the information object P is not used, the object detection unit 158 and the sum calculation unit 159 are not necessary. Therefore, the object detection unit 158 and the sum calculation unit 159 are not essential components of the present technology.

[0171] The present technology can also be applied to card games other than TCGs, such as playing cards, Hanafuda cards, Karuta, and UNO. In such cases, it is necessary to have the card identification unit 154 learn in advance the types of cards used in those card games. Therefore, the present technology can also be applied to games in which playing cards with picture cards are stacked on top of each other. It can also be applied to games such as poker, which use multiple playing cards. In such games using playing cards, the information object P may be something like a coin indicating the amount to be bet. In such games using playing cards, since any card may be placed under another card, the second card identification unit 156 needs to learn in advance about all cards.

[0172] This technology can also be applied to games other than card games, such as board games and table games like mahjong. In such cases, it is necessary to have the card identification unit 154 learn in advance the types of pieces, cards, items, tiles, etc. used in those games.

[0173] The present technology can also be configured as follows. (1) Detecting the position of the card in an image captured by a camera of an area where the game cards are placed; generating a cropped image by cropping an area of the card from the image based on the position of the card; Acquire the game situation based on the clipped image A program that causes a computer to execute an information processing method. (2) The program according to (1), wherein the game situation is the type of the card. (3) The program according to (2), wherein the type of the card in the cutout image is identified by a first identification process and a second identification process. (4) The program according to (3), wherein the first identification process identifies the type of the card in all of the cutout images. (5) The program according to (3) or (4), wherein the second classification process identifies the type of the card in the cutout image for which the type of the card could not be identified in the first classification process. (6) The card whose type could not be identified is a card placed under another card, The program according to (5), wherein the second identification process identifies the type of the card based on identification information of the exposed area of the card placed under the other cards. (7) The program according to any one of (2) to (6), which generates an output result for the latest image based on the identification result of the type of the card in the latest image and the output result for the past image. (8) The program according to any one of (1) to (7), wherein the game status is the number of cards. (9) The program according to any one of (1) to (8), wherein the game situation is a type of object that indicates information about the game. (10) The program according to (9), wherein a process is performed to detect an object showing information about the game from the cutout image in which the type of the card has been identified in the first identification process. (11) The program according to (10), wherein the information about the game is a numerical value in the game. (12) The program according to (11), wherein the sum of the numerical values indicated by all the objects detected in the same clipped image is calculated. (13) The program according to any one of (1) to (12), wherein keystone correction is performed on the image before detecting the position of the card. (14) The program according to any one of (1) to (13), which stops processing when movement of the camera is detected. (15) The program according to any one of (1) to (14), wherein processing is stopped when a foreign object is detected in the space between the surface on which the card is placed and the camera. (16) If the foreign object is a hand, the detection information of the hand is output to the outside. (17) Detecting the position of the card in an image captured by a camera of an area where the game cards are placed; generating a cropped image by cropping an area of the card from the image based on the position of the card; Acquire the game situation based on the clipped image Information processing methods. (18) a card position detection unit that detects the position of a card in an image captured by a camera within an area where a game card is placed; a cutout unit that cuts out an area of the card from the image based on the position of the card to generate a cutout image; a situation acquisition unit that acquires the situation of the game based on the cut-out image; An information processing device comprising: [Explanation of symbols]

[0174] 106···Camera 100 Information processing device 152 Card position detection unit 153 Cutout 154 Card identification unit 158...Object detection unit

Claims

1. Detecting the position of the card in an image captured by a camera of an area where the game cards are placed; generating a cropped image by cropping an area of the card from the image based on the position of the card; Acquire the game situation based on the clipped image A program that causes a computer to execute an information processing method.

2. The game situation is the type of the card. The program according to claim 1.

3. The first and second identification processes identify the type of the card in the cutout image. The program according to claim 2.

4. The first identification process identifies the type of the card in all of the clipped images. The program according to claim 3.

5. The second classification process identifies the type of the card in the cutout image for which the type of the card could not be identified in the first classification process. The program according to claim 3.

6. The card whose type could not be identified is a card placed under another card, The second identification process identifies the type of the card based on the identification information of the exposed area of the card placed under the other cards. The program according to claim 3.

7. Generate an output result for the latest image based on the identification result of the type of the card for the latest image and the output result for the past image. The program according to claim 2.

8. The game situation is the number of cards. The program according to claim 1.

9. The game situation is a type of object that indicates information about the game. The program according to claim 1.

10. A process for detecting an object showing information about the game is performed on the cutout image in which the type of the card has been identified in the first identification process. The program according to claim 9.

11. The information about the game is a numerical value in the game. The program according to claim 10.

12. Calculating the sum of the values represented by all the objects detected in the same cropped image The program according to claim 11.

13. A keystone correction is applied to the image before detecting the position of the card. The program according to claim 1.

14. Stop processing when camera movement is detected The program according to claim 1.

15. If a foreign object is detected in the space between the surface on which the card is placed and the camera, the process is stopped. The program according to claim 1.

16. If the foreign object is a hand, the detection information of the hand is output to the outside. The program according to claim 15.

17. Detecting the position of the card in an image captured by a camera of an area where the game cards are placed; generating a cropped image by cropping an area of the card from the image based on the position of the card; Acquire the game situation based on the clipped image Information processing methods.

18. a card position detection unit that detects the position of a card in an image captured by a camera within an area where a game card is placed; a cutout unit that cuts out an area of the card from the image based on the position of the card to generate a cutout image; a situation acquisition unit that acquires the situation of the game based on the cut-out image; An information processing device comprising:

Images