Game progress assisting system, game progress assisting program, game progress assisting method, and amusement facility

Abstract

To provide a game progress assisting system, a game progress assisting program, a game progress assisting method, and an amusement facility capable of assisting progress of a game using a combination of game objects.SOLUTION: A game progress assisting system assisting a game to be progressed by using a game article disposed in a field comprises: a camera for imaging a field; a projector for projecting a projection image to the field; and an information processing device. The information processing device acquires object information including a position and a type of an object corresponding to a game article on the basis of a captured image of the camera. The information processing device also generates a projection image including a first combination display showing a combination by determining the combination that satisfies the first combination condition from a combination of one of first objects positioned in a first area within the captured image and one of second objects positioned in a second area within the captured image on the basis of object information.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present disclosure relates to a system, a program, a method, and an entertainment facility that assists in a game that progresses using game items placed on a field. [Background technology]

[0002] Conventionally, board games and card games have been known in which the game progresses by combining multiple game objects to add up victory points or activate specific effects (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 07-100268 Summary of the Invention [Problem to be solved by the invention]

[0004] In a game in which there are multiple ways to combine game objects as described above, there is a need for technology that supports the appropriate progression of the game. [Means for solving the problem]

[0005] In view of the above, the following configuration example can be given.

[0006] (Configuration 1) Configuration 1 is a game progress assistance system that assists in a game that is progressed using game items placed on a field, and includes a camera that captures an image of the field, a projector that projects a projection image onto the field, and an information processing device. The information processing device has object information acquisition means that acquires object information including at least position information and type information of the object based on object images corresponding to the game items included in the captured image captured by the camera, first combination determination means that, based on the object information, determine a combination of any first object located in a first area in the captured image and any second object located in a second area in the captured image that satisfies a predetermined first combination condition, and projection image generation means that generates a projection image including a first combination display that shows the combination that satisfies the first combination condition.

[0007] According to the above configuration example, an image showing combination candidates of objects (game items) that satisfy the first combination condition is projected onto the field. The image allows the player to visually and intuitively understand how to combine game items, thereby assisting the player in playing the game appropriately. For example, this can help a beginner player who is not familiar with how to combine game items to play the game smoothly.

[0008] (Configuration 2) In configuration 2, in configuration 1, the information processing device may include a detection means for detecting, based on the position information, that each of the objects included in a combination of the first object and the second object determined to satisfy the first combination condition has been moved to a third area in the captured image. Furthermore, when it is detected that the object has been moved to the third area, the projection image generation means may generate a projection image including a third area display related to the third area.

[0009] According to the above configuration example, by generating a projection image when a game item is placed in the third area, it is possible to more appropriately assist the progress of the game, for example, by helping the player memorize how to combine game items, or by making it easier for the player to decide what action to take next.

[0010] (Configuration 3) In configuration 3, in configuration 1, the game may be a game in which multiple players compete against each other, and the first region may be associated with a first player. The combination determination means may determine, based on information about the objects, a combination that satisfies a first combination condition from among combinations of any fourth object located in a fourth region of the captured image, which is an region associated with a second player, and any second object located in the second region. Furthermore, the projection image generation means may generate a projection image including: a first combination display that indicates a combination that satisfies the first combination condition from among combinations of any first object located in the first region and any second object located in the second region; and a first combination display that indicates a combination that satisfies the first combination condition from among combinations of any fourth object located in the fourth region and any second object located in the second region.

[0011] According to the above configuration example, in a game in which multiple players compete against each other, a first combination display can be presented according to each player, thereby enabling more appropriate assistance in the progress of the game.

[0012] (Configuration 4) In configuration 4, in configuration 2, the game is a game in which multiple players compete against each other while taking turns, and the first and third regions may be associated with a first player. Furthermore, the combination determination means may determine, based on object information, a combination of any fourth object located in a fourth region of the captured image, which is an region associated with a second player, and any second object located in the second region, which satisfies a first combination condition. The detection means may detect, based on the position information, that each of the objects included in the combination of the second object and the fourth object determined to satisfy the first combination condition has been moved to a fifth region in the captured image, which is an region associated with the second player. When it is detected that a game item has been moved to the fifth region, the projection image generation means may generate a projection image including a fifth region display for the fifth region.

[0013] According to the above configuration example, it is possible to provide appropriate support for game progress even in a game played by multiple players in which game items are moved between multiple areas.

[0014] (Configuration 5) In configuration 5, in configuration 4, the game may be a game in which a plurality of players compete against each other while taking turns. The projection image generation means may be switchable between a first generation mode in which the projection image is generated by including in the projection image a 1P-side combination display that is a first combination display indicating a combination of a first object and a second object that satisfies a first combination condition, and not including in the projection image a 2P-side combination display that is a first combination display indicating a combination of a fourth object and the second object that satisfies the first combination condition, and a second generation mode in which the projection image is generated by not including in the projection image the 1P-side combination display, but including the 2P-side combination display, and may switch the generation mode from the first generation mode to the second generation mode when movement of an object from the second area to the third area is detected, and may switch the generation mode from the second generation mode to the first generation mode when movement of an object from the second area to the fifth area is detected.

[0015] According to the above configuration example, a first combination image corresponding to each player can be projected in accordance with the player's turn. This allows the player to easily understand that it is their turn and the actions they can take, and supports the progress of the game at a more appropriate timing.

[0016] (Configuration 6) In configuration 6, in any one of configurations 2 to 5, the captured image may include a second combination condition area including a presentation of a second combination condition related to a combination of multiple types of objects. The projection image generation means may generate a projection image including a second combination display that associates each of third objects arranged in the third area with a presentation of a second combination condition associated with the third object among the presentations of the second combination conditions included in the second combination condition display area.

[0017] According to the above configuration example, for example, information indicating the relationship between the object in the third area and the second combination condition can be projected onto the field and visually presented to the player, thereby helping the player to recognize the second combination condition.

[0018] (Configuration 7) In configuration 7, in configuration 6, the information processing device may further include a second condition pattern information acquisition means for acquiring position information of a second condition pattern in the captured image based on an image of a second condition pattern that is included in the field pattern and presents a second combination condition, and a field information acquisition means for acquiring position information of the field in the captured image based on the position information of the second condition pattern. The projection image generation means may generate a projection image based on the position information of the field.

[0019] According to the above configuration example, the position information of the field can be obtained using the second condition pattern, and a projection image can be generated based on this position information, so there is no need to provide a separate marker for obtaining the position information.

[0020] (Configuration 8) In configuration 8, in configuration 1 or 2, the projection image generation means may determine the placement position of the first combination display in the projection image based on position information of each of the objects included in the combination that satisfies the first combination condition, and generate the projection image.

[0021] According to the above configuration example, the positional relationship of the game items that satisfy the first combination condition is displayed in a visually emphasized manner, so that the game items that satisfy the first combination condition can be more intuitively grasped.

[0022] (Configuration 9) Configuration 9 is based on configuration 8, and the first combination display may include a line connecting each of the frame displays corresponding to the positions where each of the objects that satisfy the first combination condition is arranged.

[0023] (Configuration 10) Configuration 10 may be the same as configuration 8, wherein the game items corresponding to the first objects are arranged in a row along a predetermined direction in an area on the field corresponding to the first region, and the game items corresponding to the second objects are arranged in at least two rows along the predetermined direction in an area on the field corresponding to the second region. The projection image generation means may generate the first combination display to include a rectangular frame display surrounding each of the objects that satisfy the first combination condition, and a line whose starting point is a predetermined position on the outer edge of the frame display surrounding the first object located in the first region, on the second region side, and whose ending point is a position within the frame display surrounding the second object, among combinations of objects that satisfy the first combination condition.

[0024] (Configuration 11) In configuration 11, in configuration 10, the frame display may be a rectangular frame display. Also, the line may be a straight line starting from a predetermined position on a side of the frame display on the second region side and ending at a center position of the frame display surrounding the second object.

[0025] According to the above configuration example, an image can be projected that connects game items that satisfy the first combination condition. In particular, game items corresponding to the second object are arranged in a two-dimensional array, and by setting a position within the frame display, for example, the center position, as one end of a line, the ends of the lines in such a two-dimensional array can be spaced apart to a certain extent, making it easier to distinguish which game items satisfy the first combination condition. This makes it easier for the player to intuitively understand which game items satisfy the first combination condition.

[0026] (Configuration 12) In configuration 12, in configuration 1 or 2, the game items are card-type game items each having a different design on at least one side, and the object information acquisition means may acquire the type information based on the design of the game item.

[0027] According to the above configuration example, it is possible to support appropriate game progress even in a complex game in which the combinations are diverse, with each game having a different design. In particular, in a game using game objects in which the combination method is determined only by the design, without using letters to identify the individual game objects, projecting the above-described first combination image can more clearly present how to combine the game objects and also help the player memorize the combination methods. Furthermore, using flat game objects such as cards makes it easier to obtain object information, enabling more appropriate support for game progress. [Effects of the Invention]

[0028] According to the present disclosure, in a game that progresses using game items placed on a field, it is possible to appropriately support a player's progress in the game. [Brief explanation of the drawings]

[0029] [Figure 1] 1 is a schematic diagram showing an overall image of a system according to an embodiment of the present invention; [Figure 2] An example of a playmat design [Figure 3] A diagram to explain the list of roles [Figure 4] Hardware configuration of PC2 [Figure 5] An example of a playmat board [Figure 6] An example of the first guide image [Figure 7] An example of a playmat board [Figure 8] An example of the second guide image [Figure 9] An example of the second guide image [Figure 10] An example of the third guide image [Figure 11] Examples of second and third guide images [Figure 12] An example of the play mat with the projected image [Figure 13]A memory map showing an example of various data stored in the memory unit 22 of the PC 2. [Figure 14] An example of the data structure of game status management data [Figure 15] An example of an image for detection processing [Figure 16] Flowchart showing details of game progress assistance processing [Figure 17] Diagram explaining prohibited areas [Figure 18] FIG. 1 is a diagram for explaining stabilization processing; DETAILED DESCRIPTION OF THE INVENTION

[0030] An embodiment will be described below. FIG. 1 is a schematic diagram showing an overall view of a game progress assistance system according to this embodiment. The game progress assistance system may be provided, for example, as an entertainment facility. For example, it may be provided as one of the attractions in a theme park. It may also be provided as one of the teaching materials in educational facilities such as schools. It may also be used, for example, by individual users at home. The game progress assistance system is constructed, for example, in a specific room, and includes the elements shown in FIG. 1. FIG. 1 shows a first player (hereinafter, 1P), a second player (hereinafter, 2P), a play mat 10, game items 11, a camera 27, a projector (hereinafter, PJ) 28, and an information processing device 2.

[0031] The play mat 10 is used as a game field for playing the game, and is placed on the floor or a table, for example. Player 1 and Player 2 sit facing each other across the play mat 10. Game items 11 are placed on the play mat 10. The game items 11 are physical game elements.

[0032] Furthermore, the camera 27 and the PJ 28 are installed, for example, on the ceiling of the room. More specifically, the camera 27 is installed in a position that allows it to capture the area around the play mat 10. For example, it is installed in a position that is almost directly above the center of the play mat 10. The PJ 28 is installed in a position that allows it to include the area around the play mat 10 in its projection range. For example, the PJ 28 is installed in a position that is almost directly above the play mat 10, such as a position adjacent to the camera 27.

[0033] The information processing device 2 is, for example, a personal computer (hereinafter referred to as PC). The PC 2 is connected to the camera 27 and the PJ 28. The connection may be wired or wireless. The PC 2 may be located in the same room as the camera 27 and the PJ 28, or in a different room. The PC 2 acquires an image captured by the camera 27, performs processing based on the captured image as described below, and outputs image data to be projected to the PJ 28 based on the processing results. Details of the processing in the PC 2 will be described later.

[0034] In this embodiment, an example will be described in which the camera 27, PJ28, and PC2 are separate devices, but in other embodiments, the camera 27, PJ28, and PC2 may not be separate entities, and some or all of them may be configured as the same information processing device (housed in the same housing).

[0035] [About game items] Next, the game item 11 will be described. In this embodiment, a card-shaped game item 11, each with a different picture on at least one side, will be described as an example. More specifically, in this embodiment, "hanafuda" cards will be described as an example of the game item 11. In addition, since this embodiment uses hanafuda cards as an example, the cards (hereinafter, individual cards will be referred to as cards) are divided into 12 types (months), with four cards belonging to each type (48 cards in total). Furthermore, each card has a different picture on one side (front) and the same picture on the other side (back / rear). In this embodiment, a game format is assumed in which cards are placed on a play mat 10 and played. In addition, when referring to each card below, the type will be indicated by an alphabet, and the individual cards belonging to each type will be indicated by a number. For example, four cards belonging to type A will be referred to as cards A1, A2, A3, and A4.

[0036] [About the playmat] Next, the designs printed on the play mat 10 will be described. Figure 2 shows an example of the play mat 10. The play mat 10 is printed with a number of rectangular frames (placement frames) that indicate the placement locations of the cards. The size of the placement frames is approximately the same as the size of the cards. In the example of Figure 2, a total of 12 placement frames are printed near the center of the play mat 10. These placement frames are printed in two columns of six frames each, one above the other. The area near the center of the play mat that contains these 12 placement frames (the area surrounded by dotted lines in Figure 2) will be referred to below as the "table card area." Some Hanafuda cards are placed within the table card area as "table cards."

[0037] In this embodiment, the areas above and below the tableau card area are called the "hand area." In the hand area, some of the 48 Hanafuda cards are placed as "hands." In this example, since there are two players, the lower hand area in FIG. 2 is called the hand area for 1P (hereinafter referred to as the 1P hand area), and the upper hand area is called the hand area for 2P (hereinafter referred to as the 2P hand area). In addition, in the example of FIG. 2, placement frames similar to those described above are printed as a guide for where to place cards in the hand area. Specifically, each hand area is printed with a layout in which six placement frames are lined up horizontally.

[0038] The area to the left of the tableau pile area is called the "stock area." In the stock area, some of the Hanafuda cards are placed as a stock. In addition, two of the above-mentioned placement frames are printed in the stock area.

[0039] Additionally, the area to the right of the 1P hand area is called the "1P Torifuda Area" where 1P places the cards they have acquired as "torifuda." 24 placement slots are printed in the 1P Torifuda Area (arranged 4x6). Additionally, the area to the left of the 2P hand area is called the "2P Torifuda Area" where cards to be taken are also placed. 24 placement slots are also printed in the 2P Torifuda Area.

[0040] Note that in Figure 2, the dotted lines indicating the above-mentioned areas are shown for convenience of explanation and are not actually printed. Also, the design of the above-mentioned placement frame itself is one example, and in other embodiments, for example, the placement frame may have no border and a picture with only the interior color changed may be printed as the placement frame. Also, the placement frame may have no border and only the tableau card area where the cards are placed may be separated, and the design may be compatible with a system that recognizes the cards in the area. In other words, any picture may be used as long as it serves as a guide for where to place the cards.

[0041] In addition, in FIG. 2, a "royaku list" is printed on the upper right and lower left of the play mat 10. The "royaku" list is printed with images showing multiple types of "royaku" in a list format. The "royaku" defines a specific combination of multiple cards. In other words, the "royaku list" presents the types of cards defined as part of a specific hand. FIG. 3 shows an enlarged view of the "royaku" list. FIG. 3 shows a total of six types of "royaku," including four "royaku" consisting of three cards and two "royaku" consisting of two cards. The specific card images that make up each hand are called "individual handaku images." For example, the individual handaku image in the upper left of FIG. 3 indicates a hand consisting of a combination of three cards (card A1, card B1, and card C1, each with a different image). In the example of Hanafuda cards, this would be a "royaku" such as "Ino-Shika-Cho." Although not shown in the figure, each individual handaku image may have a character string indicating the name of the "royaku" added, for example, above it.

[0042] In addition, the hand list is printed in two places, the top right and bottom left of the play mat 10 in Figure 2, but the printed content itself is the same. However, one is printed in a direction rotated 180 degrees from the other. The hand list is not limited to being printed in the bottom right and bottom left, and the number of places is not limited to two. For example, it may be printed in four places in the four corners. For example, it may be printed in one place in the center. Furthermore, the content and display format written in each of the hand lists present in multiple places may be different.

[0043] [Hardware configuration of information processing device] Next, the hardware configuration of the PC 2 will be described. Fig. 4 is a block diagram showing the hardware configuration of the PC 2. The PC 2 includes a processor 21. The processor 21 is an information processing unit that executes various types of information processing executed in the PC 2. The processor 21 executes various types of information processing by executing information processing programs stored in a storage unit 22. The storage unit 22 may be, for example, an internal storage medium such as a flash memory or a DRAM (Dynamic Random Access Memory), or may be configured to use an external storage medium, etc.

[0044] PC2 also includes a camera communication unit 24 for communicating with camera 27. Images captured by camera 27 (hereinafter referred to as camera images) can be acquired from camera 27 connected via the camera communication unit 24. PC2 also includes a projector communication unit 25. PC2 outputs data of a projection image to be projected by PJ28 connected via the projector communication unit 25.

[0045] The PC 2 also includes an input device communication unit 23 for performing wired or wireless communication with various input devices 26 such as a keyboard and a mouse.

[0046] [An example of expected gameplay flow] Next, a brief outline of the game using the Hanafuda cards (hereinafter referred to as the Hanafuda game) assumed in this embodiment will be given. In this game, as a preparation for starting the game, 12 cards are placed in the above-mentioned tableau card area. At this time, the placement position of each card can be determined using the placement frame as a guide, but it is not necessary to place them exactly in the placement frame. Furthermore, 1P and 2P each have six cards in their hands, which are placed in the 1P hand area and the 2P hand area, respectively. The remaining cards are placed face down in a stack in the frame on the left side of the deck area. Once the above placement is complete, the game can begin.

[0047] In this game, the turn of 1P and 2P alternates as the game progresses. First, it is 1P's turn. If the combination of any one card in 1P's hand (hereinafter referred to as 1P's hand) and any one card on the tableau satisfies a predetermined condition (hereinafter referred to as the first combination condition), 1P can acquire the set of cards related to that combination. 1P places the acquired cards in the 1P's card collection area. In this example, the first combination condition is that the cards must be of the same type (same month). On the other hand, if there is no combination of cards that satisfies the first combination condition, one of the cards in 1P's hand is placed in the tableau card area as a card on the tableau card.

[0048] After completing the above steps, Player 1 flips over one card from the deck and places it in the slot to the right of the deck. If this card and any of the cards on the table satisfy the first combination condition, Player 1 can also acquire the card in that combination.

[0049] After the above steps, the turn moves to 2P. Like 1P, 2P first determines whether the cards in 2P's hand and the cards on the table satisfy the first combination condition, and if they do, they take the cards and place them in 2P's take-card area. Then, as above, they determine the first combination condition for the deck and the cards on the table, and if they satisfy the condition, they take the cards and place them in 2P's take-card area. Then, the turn returns to 1P.

[0050] The above process is repeated until the deck is exhausted, and the player with the highest score ultimately wins. Regarding the calculation of points, each card is assigned a predetermined score, and the scores of the cards acquired by each player are added together. Furthermore, if the cards acquired by each player complete the above-mentioned "yaku," the score corresponding to that "yaku" is also added. In this embodiment, the combination of cards associated with each of the above-mentioned "yaku" is defined as a second combination condition. That is, if the cards acquired complete one of the "yaku" listed in the list of "yaku," the second combination condition is determined to be satisfied. In other words, the list of "yaku" presents information about the second combination condition to the player. Furthermore, the second combination condition may be determined for each "yaku" or for multiple "yaku" together. For example, if two types of "yaku" are completed, the second combination condition may be determined to be satisfied at the time each "yaku" is completed. Also, for example, when multiple types of winning combinations are completed, they may be compared, and the winning combination that is most advantageous to the player whose turn it is in the game may be determined to satisfy the second combination condition. Also, for example, when three or more winning combinations are completed, they may be compared, and multiple winning combinations that satisfy a predetermined condition may be determined to satisfy the second combination condition.

[0051] The game ends when the deck of cards is exhausted. In this case, the winner is determined by the points based on the cards taken at that point.

[0052] [Overview of processing in this embodiment] When the Hanafuda game described above is played with the above-described configuration, the following process is executed in this embodiment. First, the camera 27 captures an image of the play mat. The PC2 analyzes the camera image using object detection AI and distinguishes between areas within the camera image, such as the hand area and the tableau card area. Furthermore, the PC2 detects the position and type of each card based on the design of each card placed within each area. Based on the detection results, the PC2 determines whether or not a card combination is possible, i.e., the first and second combination conditions. Based on the determination results, the PC2 outputs data of an image (hereinafter, a guide image) that guides the player on how to combine cards to the PJ28. Details of the information processing in the PC2 will be described later. The PJ28 then projects the guide image. As a result, a guide image showing, for example, obtainable card combinations is projected onto the play mat. This assists the player in progressing through the game according to this embodiment. For example, by projecting the guide image onto the play mat, even beginners who are unfamiliar with the rules of Hanafuda can understand which cards they should acquire, allowing for smooth game progression. It can also help players memorize the roles.

[0053] [Example of guide image] Next, an example of the above-mentioned guide image (a method for assisting the progress of the game) will be given. In this example, the following guide image is projected.

[0054] [Image showing the first combination] First, a guide image (hereinafter referred to as the first guide image) is projected to guide players regarding cards that satisfy the first combination condition. That is, the first guide image is a guide image that shows possible combinations of cards that can be obtained with cards in hand and cards on the table. FIG. 5 shows the board state of the play mat 10 at a predetermined time. FIG. 6 shows an example of the first guide image projected onto the board state of FIG. 5. Note that in FIG. 6, for ease of explanation, parts on the play mat that are not directly related to the projection of the first guide image are shown with thin lines.

[0055] In FIG. 6, the first guide image shows a rectangular frame image (hereinafter referred to as the first guide frame image) and a straight line image connecting two predetermined frame lines (hereinafter referred to as the first guide straight line image). The first guide frame image is an image for highlighting predetermined cards in the 1P hand area, 2P hand area, and tableau card area. For example, the frame image is a frame line of a size that surrounds the outer frame of the card. The color of the frame line can be any color, but here we will explain the case where white is used as an example. The first guide frame image provides a visual effect that makes the area around the predetermined card glow.

[0056] In addition, in FIG. 6, the first guide line image is an image for showing combinations of cards that satisfy the first combination condition to each of 1P and 2P. The first guide line image presents the relationship between cards that satisfy the first combination condition by connecting them with a line. If a certain card on the table satisfies the first combination condition with both one of the cards in 1P's hand and one of the cards in 2P's hand, images of two lines are projected connecting the card on the table with each of the cards in 1P's hand and 2P's hand that satisfy the first combination condition (see, for example, the first card from the top left to the right of the card on the table in FIG. 6). The first guide line image may be displayed as an animation in which a line moves from the hand to the card on the table, connecting them. Alternatively, an animation in which an arrow moves instead of a line may be displayed.

[0057] By projecting the first guide image as described above, it is possible to make it appear as if a specific card in each player's hand area and a specific card in the tableau card area are highlighted on the play mat. Furthermore, the first guide line image allows players to visually grasp the combinations of cards that satisfy the first combination condition. In the example of FIG. 6, Player 1 can grasp that three cards from Player 1's hand can be combined with cards on the tableau to obtain a card. Player 2 can also grasp that two cards from Player 2's hand can be combined with cards on the tableau to obtain a card. Therefore, even beginners who do not remember the types of cards can visually and intuitively grasp which combinations of cards they can obtain. As a result, players can reduce the time they spend thinking about how to take cards, and smooth game progress can be expected.

[0058] In this embodiment, the control is such that the first guide image as described above is always projected regardless of the player's turn. In other words, the control is such that the arrangement status of each card on the play mat 10 is grasped, and the process of simply projecting a first guide image (and second and third guide images described below) according to that arrangement status is repeated at a predetermined interval (for example, 30 fps). In this regard, the projection of the first guide image for 1P and the first guide image for 2P may be switched depending on the player's turn. For example, when it is 1P's turn, the control may be such that the first guide image for the 1P hand area and the tableau card area is projected, but the first guide image for 2P is not projected. On the other hand, when it is 2P's turn, the control may be such that the first guide image for 1P is not projected, but the first guide image for 2P is projected, in a second projection mode. Any method may be used to determine whether the turn has changed. For example, the projection mode may be switched at the timing when it is detected that a card that satisfies the first combination condition has been moved from the tabletop card area to the player's card-taking area. For example, the projection mode may be switched from the first projection mode to the second projection mode at the timing when it is detected that a card has been moved from the tabletop card area to the player's card-taking area, assuming that the turn has been passed to player 2P. Also, the projection mode may be switched from the second projection mode to the first projection mode at the timing when it is detected that a card has been moved from the tabletop card area to the player's card-taking area, assuming that the turn has been passed to player 1P.

[0059] Here, a supplementary explanation will be given regarding the positions of both ends of the first guide line image. In the example of FIG. 6, one end of the first guide line image (called the starting point) is located at a position that contacts the upper side of the frame image in the hand area of ​​1P or 2P. Meanwhile, the other end (called the end point) is located at the center of the frame image in the tableau card area. Therefore, a visual effect of pointing to the center of each card can be provided while leaving a certain amount of space between the end points. In this embodiment, it is assumed that the hand cards are arranged in a single horizontal row, and the tableau cards are arranged in two vertical rows. In such a two-dimensional arrangement, by setting the center of the frame image on the tableau card side as the end point of the line as described above, it is possible to more clearly convey to the player which cards in the two-dimensionally arranged tableau card area are available for acquisition.

[0060] In other embodiments, when the processing of this embodiment is applied to a game other than Hanafuda, the ends of the straight line image may be straight lines connecting the sides of the frame line image, or straight lines connecting the centers of the frame line images.

[0061] [Image guiding you to the second combination condition] Next, a guide image (hereinafter referred to as the second guide image) for guiding regarding the second combination condition will be described. The second guide image is an image for guiding regarding the "roles" printed in the list of winning combinations. Prior to the description of the second guide image, first, assume that the board shown in FIG. 5 above has been changed to the board shown in FIG. 7 as a result of each player taking one turn at a time. Specifically, from the board shown in FIG. 5, 1P has placed four cards as cards to be taken: two cards on the table, one card from 1P's hand, and one card from the deck. Also, assume that 2P has placed only one card on the table and one card from 2P's hand in the card-to-be-taken area, and has drawn one card from the deck to replenish the cards on the table.

[0062] FIG. 8 shows an example of a second guide image projected onto the board surface of FIG. 7 above. Note that, for ease of explanation, in FIG. 8, portions of the play mat that are not directly related to the projection of the second guide image are shown with thin lines. The second guide image includes a frame image (hereinafter, the second guide frame image) that highlights any of the "royaku" (the individual winning combinations) printed in the winning combination list, or the printed portion of any of the cards that make up the "royaku." More specifically, the second guide image is an image that guides users to know that there is a "royaku" that can be completed when there are cards that make up a predetermined winning combination in the torifuda area, and what that "royaku" specifically is. For example, when there is a card that makes up a predetermined "royaku" in the torifuda area, the second guide frame image is used to highlight the printed portion of any one of the cards in the individual winning combinations that correspond to that card. FIG. 8 is an example of a second guide frame image in such a case. Furthermore, when any of the "royaku" is completed in the torifuda area, a second guide frame image is projected that highlights the entire individual winning combination that corresponds to the completed winning combination.

[0063] Furthermore, when a card is placed in the torifuda area, if the placed card has the same pattern as a card that constitutes one of the "yaku" (hand combinations), a linear image indicating the "yaku" (hereinafter, "second guide linear image") is temporarily projected. For example, as shown in FIG. 9, a second guide image including a second guide frame image highlighting the cards that constitute the "yaku" in the hand combination list and a second guide linear image extending from the edge of the second guide frame image toward the position where the torifuda is placed is projected for, for example, a predetermined time. This allows the player to easily visually grasp that if the newly acquired torifuda is part of a "yaku," there is a possibility that the predetermined "yaku" can be completed. Note that the hand combination list is printed on the play mat in advance, so the positions where the patterns of each card that constitutes the hand are printed can also be known in advance. Therefore, if the position of the play mat can be identified in the captured image, the projection position of the second guide frame image can be derived without performing separate image recognition processing on the hand combination list.

[0064] The positions of both ends of the second guide line image may be, for example, a line starting from a position on the side of the second guide frame line image and ending at the center of the third guide frame line image. The relationship between the start and end points may also be reversed. The lines may also connect the centers of the frame line images.

[0065] [Image showing information about the cards] Furthermore, in this embodiment, a guide image (hereinafter referred to as the third guide image) that guides information about the taken cards is also projected. FIG. 10 shows an example of the third guide image projected onto the board surface of FIG. 7 above. The third guide image includes a score notation that indicates the current score and a frame image (hereinafter referred to as the third guide frame image) that highlights a specific card in the taken cards area. The score notation is projected at a position above the taken cards area. The third guide frame image is an image that highlights a card that constitutes one of the "hands" defined in the hand list when that card is present in the taken cards. In other words, the presentation of the second guide frame image and the third guide frame image provides a visual effect in which the same card is highlighted. This allows the player to intuitively understand which hand in the hand list the card highlighted in the third guide frame image constitutes.

[0066] FIG. 11 shows examples of the projection of the second and third guide images. The board shown in FIG. 11 is based on the board shown in FIG. 7, where 1P moves one card from the tabletop and one card from his hand to the take-card area and places one card from the deck on the tabletop, completing "Yaku D" in 1P's take-cards. In this case, a second guide frame image that surrounds the entire role D is projected as the second guide image at the location where "Yaku D" is printed in the role list. Furthermore, a third guide frame image that highlights the score notation and the three cards that make up "Yaku D" is projected as the third guide image in the take-card area. Furthermore, a second guide frame image that highlights some of the cards in "Yaku B" that have not yet been completed, and a third guide frame image that highlights the card in the take-card area that corresponds to that card, are also projected.

[0067] Regarding the second guide image when a winning combination is completed, after the second guide straight line image as shown in FIG. 9 above is displayed for a certain period of time, the projection content may change to surround the entire completed winning combination as shown in FIG. 11. Also, when a winning combination is completed, the projection content of the third guide frame image may be changed. For example, in order to distinguish between cards that make up a winning combination that has already been completed and cards that will make up a winning combination in the future, the projection of the third guide frame image related to the completed winning combination may be erased.

[0068] By projecting the second guide image as described above, the player can grasp which hands are close to being completed or have been completed. It can also help the player remember which cards make up which hands. Furthermore, by projecting the third guide image as described above, it is possible to provide information that will help the player when considering game strategy, etc.

[0069] FIG. 12 shows an example of the final projected guide image, including the first to third guide images. Projecting a guide image such as that shown in FIG. 12 onto the play mat 10 can help the player progress smoothly through the game. For example, the player can refer to the first guide image to quickly understand the possible cards that can be taken. Furthermore, the player can understand the current status of the cards that have been taken and whether or not there is a hand that is close to being completed, using the second and third guide images, which can be used to memorize the hand patterns of Hanafuda cards and to develop a game strategy.

[0070] The above-mentioned specific examples of various guide images are merely examples, and are not limited to the above. Images with the following contents may also be used as guide images.

[0071] For example, when there are multiple combinations that satisfy the first combination condition, the guide images are projected for all of the combinations in the above example, but it is also possible to project guide images only for some of the combinations. For example, it is also possible to project guide images only for the combination that is most advantageous to the player.

[0072] Furthermore, when indicating the existence of multiple combinations that satisfy the first combination condition, the way in which the guide image is displayed may differ for each combination. In the above example, all combinations are displayed using straight line images, but for example, if there are two combinations, one may be displayed by connecting them with a straight line, and the other may be displayed by flashing the first guide frame image. Alternatively, the color of the first guide frame image may be different for each combination. Also, only frame line images or only straight line images may be displayed.

[0073] Furthermore, the image content and expression content for indicating the first combination are not limited to the image using a straight line as described above. For example, it is also possible to display words or an image (other than a straight line) indicating the first combination condition. For example, it may be a curved line.

[0074] Also, only the portion of the card that satisfies the first combination condition may be displayed in a different color. Conversely, an image may be projected in which cards other than those that satisfy the first combination condition (i.e., cards that cannot be acquired) are filled in with a predetermined color.

[0075] The first guide image may also include the coordinates where each card is placed, or a number indicating the frame (such as a frame number). For example, a string of characters such as "You can get the nth card and the nth card" or "You can get the nth month card" may be projected as the first guide image. In the above example, the first guide frame image is used to light up the periphery of the card that can be obtained, but an image may also be projected that lights up the entire card that can be obtained.

[0076] Also, a guide image may be used that adds a predetermined effect (for example, ripples) around each card.

[0077] Alternatively, a projection that cannot be detected by humans may be used. For example, a minute perturbation or the like may be applied, and the result may be acquired by a camera other than the camera 27, and a predetermined sound may be output from a predetermined position to provide audio guidance of the first combination condition or the like.

[0078] Similarly, images of any content may be used for the second guide image and the third guide image. For example, the second guide image may project an image that highlights the entire area of ​​the winning combination list, or may highlight individual winning combinations as described above, or may highlight individual cards that make up each winning combination. When individual cards that make up each winning combination are highlighted, the player can specifically understand which cards of which "winning combinations" in the winning combination list the player has obtained. When individual winning combinations are highlighted, the player can understand what kind of winning combination the acquired cards make up. When the entire winning combination list is highlighted, the player can understand that some "winning combination" has been achieved, but the player must determine for themselves which specific winning combination has been achieved. In either case, it is possible to assist players, especially beginners, in memorizing "winning combinations" through actual gameplay.

[0079] In addition, in this example, the description is given assuming that the list of winning combinations is printed on the play mat 10 in advance, but in other embodiments, the list of winning combinations may not be printed on the play mat 10, and an image of the list of winning combinations as described above may be projected as the second guide image. In addition, in this case, the content of the image projected as the list of winning combinations may be changed depending on the state of the player's hand. For example, an image of the list of winning combinations may be projected in which winning combinations that are close to being completed are highlighted.

[0080] Furthermore, with regard to the third guide image, instead of or in addition to displaying the points, for example, text or images indicating the placement time of each card placed in the taken cards area may be projected. This is useful for games that use the time it takes to place cards as a game element, such as a time race. Also, various information regarding the taken cards may be projected as the third guide image. For example, the total number of taken cards currently in play, the number of cards by type (month), the number of completed hands, etc. Furthermore, when applied to games other than Hanafuda, for example, an image or the like that guides the player on actions that can be taken based on the game items placed in the area corresponding to the taken cards area may be projected. Also, for example, other parameters used during the game, such as the number of remaining actions or life, may be projected.

[0081] [Details of the process performed on PC2] Next, with reference to FIGS. 13 to 18, the process executed by the PC 2 to project the guide image described above (hereinafter, game progress assist process) will be described in detail.

[0082] [About data usage] First, the various data used in the game progress assistance process will be described. Fig. 13 is a memory map showing an example of the various data stored in the memory unit 22 of the PC 2. The memory unit 22 stores a progress assistance process program 301, an object detection model 302, play mat definition data 303, camera image data 304, detection process image data 305, stabilization process data 306, object data 307, game status management data 308, and PJ output image data 309.

[0083] The progress assistance processing program 301 is a program for performing the game progress assistance processing described above, specifically, a program for executing the processing shown in the flowchart of Fig. 16, which will be described later.

[0084] The object detection model 302 is a trained model that has been trained in advance on the designs of each Hanafuda card, and is used in image recognition processing (object detection algorithm) by object detection AI. In this example, Hanafuda cards are detected in the image by inputting image data of an image for detection processing, which will be described later. One or more bounding boxes (hereinafter referred to as BBoxes) are output as the detection result. An image of the detection result is shown in FIG. 15. In FIG. 15, the image for detection processing, which will be described later, appears to have multiple rectangular frames (BBoxes) on it. In this example, if there are no false detections, one BBox basically corresponds to one face-up card. Note that in this example, face-down cards are not subject to detection. As a variant, for example, the design of a face-down card may be detected but not reflected in the output to the BBox. Each BBox contains at least information indicating the type of card and the position of the card in the image.

[0085] The play mat definition data 303 defines information about the play mat 10. Specifically, the following information is defined. First, the overall physical size of the play mat 10 is defined. Also, information specifying the positions and ranges on the play mat 10 to be treated as the hand area, tableau card area, taken card area, and deck area is defined. Furthermore, image data showing the images of the hand lists and information indicating the printing positions of the two hand lists on the play mat 10, in other words, information specifying the areas where the hand lists are printed, are defined. Also, information indicating the printing positions of the images of each card printed in the hand lists is defined. Also, information indicating the "prohibited area" described below is defined.

[0086] The camera image data 304 is stored image data captured by the camera 27.

[0087] The image data for detection processing 305 is image data obtained by processing the camera image data 304, and is image data used as input to the object detection model 302. The details of the processing will be described later.

[0088] The stabilization processing data 306 stores various data used in the stabilization processing described below.

[0089] The object data 307 is data of an "object" generated based on a BBox, which is the detection result by the object detection model 302. The object corresponds to each card placed on the play mat 10. In addition, in the Hanafuda game process described below, various game processes such as determining card combinations are performed based on the object. In other words, the object is data that represents each card, which is a physical game item, and can be used in the game process. In addition, as will be described in detail later, in this embodiment, a stabilization process described later is performed on the detection result by the object detection model 302. Then, an object is generated based on the detection result (BBox) that reflects the result of the stabilization process, and stored as object data 307. Basically, one object corresponds to one card. However, this does not apply to merge-type objects described below. In addition to the data inherited from the BBox, each object includes various data used in the game process, such as "detection parameters" described below.

[0090] The game status management data 308 includes various data for managing the progress of the Hanafuda game according to this embodiment. Fig. 14 shows an example of the data configuration of the game status management data 308. The game status management data 308 includes the following data:

[0091] First, the combination definition data 321 includes the first and second combination conditions.

[0092] Based on the position information of the objects (each card), the following data is also stored as data classifying each object into one of hand cards, cards on the table, cards in the deck, or cards taken. That is, 1P hand data 322, which is data corresponding to each card in the 1P hand area, and 2P hand data 323, which is data corresponding to each card in the 2P hand area, are stored. Also stored are 1P taken card data 324, which includes information corresponding to each card taken by 1P and information indicating the winning combination corresponding to the card taken, and 2P taken card data 325, which includes information corresponding to each card taken by 2P and information indicating the winning combination corresponding to the card taken. Also stored are card table data 326, which is data corresponding to cards placed in the card table area, and card deck data 327, which is data corresponding to cards placed in the deck area.

[0093] Furthermore, 1P point data 328 and 2P point data 329 for managing 1P and 2P points are also stored.

[0094] Also stored are first guide image data 330, which is image data for the first guide image, second guide image data 331, which is image data for the second guide image, and third guide image data 332, which is image data for the third guide image.

[0095] Also stored is arrangement status history data 333 for holding data from, for example, several frames to several tens of frames back as history for the 1P hand data 322, 2P hand data 323, 1P taken cards data 324, 2P taken cards data 325, tableau cards data 326, and deck data 327. In other words, the arrangement status history data 333 is data that temporarily stores the arrangement status of each card from several frames to several tens of frames back from the current frame.

[0096] 13, the PJ output image data 309 is image data generated based on the first guide image data 330, second guide image data 331, and third guide image data 332, and is data of the guide image that is ultimately projected onto the PJ 28. Furthermore, these guide images may be placed in one virtual space and generated as one piece of image data.

[0097] [Details of the processing performed by Processor 21] Next, the game progress assistance process according to this embodiment will be described in detail. Note that the flowchart shown below is merely an example of the processing steps. Therefore, the processing order of each step may be changed as long as the same results are obtained. Furthermore, the values ​​of variables and thresholds used in the determination steps are merely examples, and other values ​​may be used as necessary.

[0098] 16 is a flowchart showing the details of the game progress assistance process according to this embodiment. Note that the processing loop of steps S2 to S9 in this flowchart is repeatedly executed for each frame. In addition, this embodiment will be described assuming that the frame rate is 30 fps.

[0099] [Preparation] First, in step S1, processor 21 executes preparation processing for starting a game. Processor 21, for example, causes PJ 28 to project a message instructing the player to place cards in predetermined positions on play mat 10, specifically, in each of the areas of the tableau cards, hand cards, and deck cards. The player places each card in a predetermined position on play mat 10 using the placement frames as landmarks. Processor 21 detects the cards on play mat 10 using a process similar to the detection process described below based on camera image data 304, and determines whether the cards have been placed in the predetermined positions. Once the cards have been placed in the predetermined positions, processor 21, for example, projects a predetermined game start message, and prompts the player to start playing the game.

[0100] [Get camera image] Next, in step S2, the processor 21 acquires the camera image data 304 sent from the camera 27.

[0101] [Identifying the playmat area] Next, in step S3, the processor 21 cuts out a portion of the play mat 10 from the camera image and generates image data 305 for detection processing. Specifically, the following processing is performed. First, the processor 21 uses the play mat definition data 303 to identify the portion of the play mat 10 in the camera image (hereinafter referred to as the play mat area). In this example, the "role list" printed in the two places is used as a marker to identify the play mat area in the camera image. That is, the processor 21 performs a predetermined image recognition process on the camera image data 304, detects the two "role lists" and identifies their positions. Since the relative positional relationship of the "role lists" on the play mat 10 is defined by the play mat definition data 303, if the positions of the two role lists are known, the position of the play mat area in the camera image can also be identified. Furthermore, processor 21 generates an image (hereinafter referred to as detection processing image, see Figure 15 above) for the identified play mat area in which the areas corresponding to the parts where the above-mentioned "list of roles" are printed are blacked out, and stores it as detection processing image data 305.

[0102] As described above, identifying the play mat area from the camera image and preparing an image with the role list section blacked out has the following advantages. The position of the play mat 10 in the camera image is identified based on the list of winning combinations, and the projection position of the image to be projected can be determined based on this position. This allows the guide image to be projected so that it follows the change in the position of the play mat 10 during gameplay. - Prevents items that have moved outside the play mat area from being mistakenly detected as Hanafuda cards. In the detection process described below, the size of the image data can be reduced compared to when the camera image is processed as is, thereby reducing the time required for the detection process. - By making the "List of Hands" part of the image black, it is possible to prevent the part of the list of hands that has Hanafuda pictures printed on it from being mistakenly detected as Hanafuda. The ratio of the physical size of the Hanafuda cards to the playmat 10 is known, so this ratio can be used in the stabilization process described below.

[0103] [Detection of Hanafuda (game items)] Next, in step S4, the processor 21 uses an object detection AI to detect each card in the camera image. That is, a process for detecting cards present on the play mat 10 is performed. Specifically, the processor 21 inputs the detection process image data 305 to the object detection model 302. As a result, an image recognition process is performed, and the BBox is output as the detection result. That is, position information and type information of each card in the camera image are obtained. Note that the position information may be represented as coordinates in the camera image, or may be information indicating a position in the real world. Furthermore, in this embodiment, the card type information is exemplified as information that uniquely identifies each card. However, in other embodiments, the type information may merely identify, for example, the "moon" in Hanafuda cards or the "suit" in playing cards, depending on the game content.

[0104] [Stabilization processing] Next, in step S5, processor 21 performs stabilization processing to stabilize the detection results obtained in step S4. This is done in consideration of the possibility that the detection results may be unstable over multiple frames depending on the placement of the Hanafuda cards, the relative positions of the player and camera 27, changes in the lighting environment, and so on. For example, Hanafuda cards may not be detected even though they are present, may be detected as present even though they are not, or may be detected as a different type of Hanafuda. Therefore, in order to stabilize the detection results, stabilization processing is performed using the detection results over multiple frames. This processing, for example, deletes inappropriate BBoxes, merges overlapping BBoxes into one, and treats cards that were temporarily not detected as still present. Then, based on the detection results (BBoxes) reflecting the results of this stabilization processing, the object is generated and stored as the object data 307. Using objects that have undergone this stabilization processing improves the accuracy of game processing and enables more appropriate game processing. For example, the stabilization processing may include the following steps.

[0105] [Examples of stabilization treatment] [Process to remove inappropriate BBoxes] The physical size of the play mat and the physical size of the Hanafuda cards are known, so any BBoxes found that are too large or too small compared to these sizes are deleted.

[0106] [Removing BBoxes near the borders of the playmat] For example, cards placed near the edge of the play mat are often partially hidden, which increases the likelihood of them being misidentified as other Hanafuda cards. Therefore, a "prohibited area" is defined within the play mat, as shown in Figure 17. Any BBoxes that overlap with the prohibited area are then deleted at this stage.

[0107] [Merging adjacent BBoxes in the same frame] In the detection results for a certain frame, it is possible that multiple BBoxes will be detected for the same Hanafuda card. In such cases, these will be treated together. For example, pairs of BBoxes with an IoU (Intersection over Union) greater than a certain value will be merged and treated as a single object (hereinafter sometimes referred to as a merged object). However, merged objects are designed to retain information about the multiple cards before they were merged. At this point, this merged object will be treated as an object that could be, for example, card A or card B.

[0108] [Merging objects that are similar when viewed across multiple frames] In the above detection process, even though the actual bill continues to exist, various factors, such as an obstacle between the camera and the bill, may prevent the BBox corresponding to the bill from being detected in a certain frame. Therefore, in this embodiment, a parameter called a "detection parameter" is introduced to the object to enhance robustness against the phenomenon in which the BBox corresponding to the object is temporarily not detected. Specifically, this process compares, for example, the object obtained as a result up to the previous frame with the object newly obtained in the current frame, and merges objects whose IoU is equal to or greater than a certain value. Furthermore, as described above, a parameter called a detection parameter is assigned to the object. Then, the detection parameter of the object (bill) detected in the current frame is increased, while the detection parameter of other objects is decreased. As a result, even if a certain bill is temporarily not detected, if the detection parameter of the object corresponding to that bill is equal to or greater than a certain value, it can be treated as still existing in the position where it was in the previous frame, even though it was not detected in the current frame. Note that for the above merge-type object, a detection parameter is assigned to each bill that may be included in the merge-type object. Then, the object can be identified as one of the bills using the following process.

[0109] [Adjustment process for Hanafuda cards that are easily misidentified] When card types that are prone to misidentification are known as advance information, the following processing is performed on the corresponding cards. For example, consider a situation in which one card A and one card B are on the tableau card area. Ideally, card A should be detected as card A and card B as card B. However, for some reason, both cards may be detected as card A. In such a case, the "card detected as card A" has an X% probability of being card A and a Y% probability (=100%-X%) of being card B. In this embodiment, in such a case, the detection parameters of the "card detected as card A" are divided into "detection parameters for card A X%" and "detection parameters for card B Y%." This improves the accuracy of the "Hanafuda type assignment process" described below. Furthermore, the two cards mentioned above both have the detection parameters for "card A" and "card B." As a result, when one of the cards is assigned as "card A" in the "Hanafuda type assignment process" described below, the other is assigned as "card B" without being deleted due to lack of an assignment destination.

[0110] In other embodiments, detection parameters may not be distributed, and a default card may be assigned to a card whose Hanafuda type to be assigned by the Hanafuda type assignment process described below has already been assigned to another card.

[0111] [Hanafuda type assignment process] Furthermore, the stabilization process assigns a Hanafuda type to each object. Since the maximum number of Hanafuda cards that can exist for each type is known, this information is used to assign a Hanafuda type to each object. If there are no other objects (BBoxes) detected as the same card, the Hanafuda type indicated in the BBox can be assigned. However, for cards that are prone to misidentification, as described above, if multiple cards are detected as the same type, the following processing is performed. For example, if the maximum number of cards for each type is 1 and two cards, "Card A" and "Card B," are both detected as the same type, "Card A," matching is performed starting with the merge-type object with the highest detection parameter for Card A. For example, suppose there are two merge-type objects containing "Card A" and "Card B," both of which are prone to misidentification. One of these objects is designated Object A and the other is designated Object B. In this case, suppose the detection parameters were distributed as shown in Figure 18 using the above-mentioned "Adjustment Process for Hanafuda Cards that are prone to misidentification." In this case, Object A, which has the higher detection parameter for Hanafuda A, is assigned the type "Card A." On the other hand, since "Card A" has already been assigned to object B, the other "Card B" will be assigned to it.

[0112] [Deletion process] Furthermore, in the stabilization process, a process is performed to delete objects whose detection parameters have become 0. In other words, a process is performed to delete objects that are not being used.

[0113] Through the various processes described above, the detection results of the object detection AI are stabilized, and then the game processing described below is carried out.

[0114] [Hanafuda game processing] 16, next, in step S6, processor 21 executes various game processes based on the rules of Hanafuda, based on the detection results that have been subjected to the stabilization process. Then, based on the results, the content of the image to be projected as the guide image is determined. The specific processing content of the game process may be any processing as long as it can generate the guide image as described above, but for example, the following processing may be performed.

[0115] (1) Situation assessment processing First, based on the detection results after the stabilization process, the processor 21 determines the objects present in each of the areas in the camera image and reflects the placement status of each card in the game status management data 308. That is, a process for grasping the placement status of cards on the play mat 10 is performed based on the camera image. Specifically, the processor 21 generates the objects based on the detection results (BBox). Furthermore, based on the range of each of the areas defined in the play mat definition data 303 and the position information of each object, the processor 21 determines what objects are placed in each area. In other words, a process for dividing the camera image into the above-mentioned areas and determining which area contains the object corresponding to each card is performed. Furthermore, the processor 21 determines whether a predetermined card has been newly placed in each area or whether cards have moved between areas, for example, by comparing the placement status of the previous frame derived from the placement status history data 333 with the current placement status. For example, when it is detected that a predetermined card has been placed in the taken card area, processor 21 determines whether the object corresponding to that card is the same as the object corresponding to the card that was placed in the hand area or the tableau card area in the previous frame. Based on this result, it can be determined that a card has been moved from the hand area or the tableau card area to the taken card area. Furthermore, for example, if the detection result indicates that the same card exists in both the tableau card area and the taken card area during processing related to the same frame, control is also performed to prioritize the card in the taken card area. In other words, control is performed to treat this card as having been moved to the taken card area. Furthermore, in addition to the timing at which a card is placed in the taken card area, for example, it may be determined that a movement to the taken card area has occurred by detecting that a predetermined card has left the hand or tableau card area. Then, based on the results of the situation understanding as described above, processor 21 appropriately updates the contents of 1P hand data 322, 2P hand data 323, 1P taken cards data 324, 2P taken cards data 325, cards on the table data 326, and deck data 327 of game situation management data 308.

[0116] (2) First combination condition determination process Furthermore, processor 21 determines whether a first combination condition between the current hand and cards on the table is met for each of 1P and 2P. Any determination method may be used to determine the first combination condition and the second combination condition described below, but in this example, the combination determination process is performed based on the detected cards in the hand and cards on the table and the combination definition data 321.

[0117] (3) Second combination condition judgment process Processor 21 also determines whether the second combination condition is met for the cards in the taken cards area. For example, when it is detected that a predetermined card has been placed in the taken cards area, it determines whether the card is a card that constitutes a predetermined hand. Furthermore, if the card is a card that constitutes a predetermined hand, it also performs processing to identify the printing position of the image of the corresponding card in the hand list based on play mat definition data 303. (4) Processing of the pick-up area Processor 21 also performs various game processes based on the game situation and the rules of Hanafuda, such as calculating points based on the cards in the taken cards area. For example, when it is detected that a predetermined card has been placed in the taken cards area, it performs a process of adding points defined for that card. It also determines whether a predetermined hand has been completed with that card, and if so, adds points according to that hand. (5) Other game processing In addition, various processes related to the Hanafuda game are performed depending on the state of the game. Note that in this example, the first guide image is constantly projected, but in other embodiments, when the projection content is changed according to the player's turn, the following process may be performed. For example, at the start of the game, control is performed using the first projection mode as described above. That is, a guide image that includes a first guide image for 1P but not a first guide image for 2P is projected. Thereafter, when it is detected that a predetermined card has been placed in the 1P taken card area, the projection mode may be switched to the second projection mode. That is, a guide image that does not include a first guide image for 1P but includes a first guide image for 2P may be projected. Further, when it is detected that a predetermined card has been placed in the 2P taken card area, the projection mode may be switched to the first projection mode. Also, in this example, an example is shown in which the second guide image and third guide image are constantly projected, but they may be controlled to be projected or not depending on the player's turn or the state of the game.

[0118] Next, in step S7, processor 21 determines the characters and images to be projected onto PJ 28 based on the position information of each object and the results of game processing such as card combination determination. Furthermore, processor 21 generates the above-mentioned first guide image data 330, second guide image data 331, and third guide image data 332 based on the determined contents. Note that with regard to each piece of guide image data, in processing for each frame, processing may be such that an image is generated from scratch based on the position information of each object, or processing may be such that image data prepared in advance is read and applied. Furthermore, processor 21 determines the position of each guide image in the projected image based on the position information of each object.

[0119] [Output of image data to be projected] Next, in step S8, the processor 21 generates PJ output image data 309 including the first to third guide images. Then, the processor 21 outputs the PJ output image data 309 to the PJ 28. As a result, a guide image including the first to third guide images is projected from the PJ 28.

[0120] [End judgment] Next, in step S9, processor 21 determines whether a predetermined condition for ending the game has been met. The condition may be any condition, but may also be determined to be ended when, for example, no Hanafuda cards are detected in any of the above-mentioned areas, there are no cards on the table, there are no cards in the deck, there are no cards in the hand, a predetermined number of turns have been played, a taken card shows a predetermined "royaku," a picture indicating the end of the game is recognized, or a specific operation has been performed. If the condition for ending the game has not been met (NO in step S9), the process returns to step S2 and the process is repeated. If the condition for ending the game has been met (YES in step S9), processor 21 ends the game progress assistance process.

[0121] In this embodiment, when a player plays a Hanafuda game using Hanafuda cards, which are actual game items, the object detection AI is used to grasp the arrangement of the cards on the play mat, and a guide image is projected onto the play mat by the PJ28 to guide the player on how to combine cards and possible combinations. This allows even beginners who are unfamiliar with the rules of the Hanafuda game to play smoothly. In other words, a guide image can be projected according to the situation on the play mat, providing more appropriate assistance in progressing through the game.

[0122] Furthermore, the game progress can be supported not only for beginners but also for intermediate and advanced players. The above-described processing can be applied not only to Hanafuda but also to card games such as trading card games. In this case, for example, when a new card is added to the card pool and the combination conditions in the game are updated, the intermediate and advanced players can easily understand and recognize the combination conditions, thereby supporting the game progress of intermediate and advanced players.

[0123] [Variations] In the above embodiment, an example is given in which a Hanafuda game is played using Hanafuda cards as game items. The game items may be, for example, playing cards or the trading cards mentioned above. Furthermore, the game items are not limited to card-type items such as Hanafuda cards, but may also be game items having a predetermined shape, such as chess pieces. In other words, the game items are not limited to card-type items as long as they are used in a game in which game items are placed on a predetermined field. Furthermore, the game using game items may also be a game in which cards and pieces are placed together. Furthermore, the sizes and shapes of the cards and pieces do not necessarily have to be the same. For example, different game items may be used for each player.

[0124] In the above embodiment, an example was given in which the progress assistance process by projecting the guide image was applied to a Hanafuda game played by two players. In other embodiments, the above process may be applied to a card game played by one player. The above process may also be applied to a game in which two players cooperate. The above process may also be applied to a game in which three or more players compete or cooperate. In the case of a game played by three or more players, the game may be played with players divided into two or more teams. In that case, cards and areas may be shared by players belonging to the same team.

[0125] Furthermore, although the above example illustrates the play mat 10 as a mat on which a predetermined picture is printed, in other embodiments, for example, an LCD monitor may be used instead of the play mat 10. That is, the above processing is also applicable to games in which predetermined game items are placed on an LCD monitor displaying a predetermined picture and played. In this case, instead of projecting a guide image from the PJ 28, an image corresponding to the guide image may be displayed on the LCD monitor. Alternatively, the play mat may be a desk or floor on which a predetermined picture is drawn. Alternatively, the picture may be projected onto the desk or floor by a projector based on a different control method. Furthermore, the picture does not have to be printed as long as it is affixed. For example, it may be hand-drawn or carved. The play mat may also be attached to a wall using a magnet, glue, or the like.

[0126] In the above example, the play mat 10 is divided into multiple areas, such as a hand area, a tableau card area, and a taken card area, for game processing. These area configurations may be configured appropriately depending on the game content. For example, two areas defined at separate locations that are not adjacent to each other may be grouped and treated as a single area in game processing. Also, for example, different areas may be configured to partially overlap. For example, the tableau card area and the hand area may be configured to partially overlap. In this case, in order to identify whether a card placed in the overlapping area is a hand card or a tableau card, the object may have information identifying the area to which it belongs. In this case, to identify whether the card is a hand card or a tableau card, a frame image may be projected onto the hand area and the tableau card area, surrounding the card with a different color, shape, or thickness depending on the area to which it belongs.

[0127] Furthermore, regardless of whether the areas overlap, each object corresponding to the above-mentioned card may have information identifying the area to which it belongs. Furthermore, in this case, by determining that a predetermined operation for changing the information has been performed, it may be treated as if an operation for moving each card between areas has been performed. For example, consider a case where an LCD monitor is used as the play mat 10. In this case, the information identifying the area to which a specific card belongs may be rewritten by a player performing a predetermined operation. In this case, although the card does not actually move physically, the display content on the LCD monitor may be changed, for example, by scrolling the display content to indicate that the card has been moved from the tableau card area to the taken card area.

[0128] Furthermore, data that classifies each object into one of hand cards, cards on the table, deck cards, and taken cards may be added to each object data as a flag indicating the area.

[0129] In other embodiments, the above process may be applied to a game that does not include some of the above areas, depending on the game content. For example, the above process may be applied to a game that does not use an area corresponding to the taken card area. In this case, the process may be such that only the first guide image is projected. The above process may also be applied to a game that includes other areas in addition to the above areas. The number of the above areas may also be different.

[0130] In the above example, a specific printed content of the winning combination list is illustrated as a design that lists the patterns of the cards of each winning combination (individual winning combination patterns). In this regard, in other embodiments, for example, a winning combination list may be used that shows the composition of each winning combination with a character string. Also, for example, a predetermined barcode may be printed instead of the winning combination list. Then, when a player reads the barcode with, for example, a smartphone, an image corresponding to the winning combination list may be displayed on the screen of the smartphone.

[0131] Furthermore, with regard to the method of recognizing the position of the play mat 10 in the camera image, the above example shows an example in which the list of winning combinations is used as a marker to identify the position of the play mat 10 in the camera image. That is, an example in which a part of the play mat 10 is used as a marker is shown. Then, based on the position of this marker and, for example, size information and position information predefined in the play mat definition data 303, the position of the play mat 10 in the camera image is identified. In this regard, in other embodiments, the entire play mat may be used as a marker to identify its position. In this case, the position of the play mat in the camera image may be directly identified from the recognized position of the play mat.

[0132] Furthermore, in other embodiments, the hand area is not limited to cards placed on the play mat 10 as described above, and the game may be played in a style in which the player holds the cards in their hands. In this case, the camera 27 may be positioned so that it can capture an image of the player's hands. The area in the camera image that captures the player's hands may then be treated as the hand area, and the various processes described above may be performed.

[0133] Furthermore, since the above example uses Hanafuda cards, the first combination condition is "cards from the same month" and the second combination condition is "hand combination." The combination conditions are not limited to these and may be any predefined combination depending on the game. For example, the first combination condition may be "cards with the same design" or "cards of the same color." Regarding the number of targets to be combined, the above example illustrates a first combination condition of two cards. However, in other embodiments, the first combination condition may be, for example, that two cards in the hand and one card on the table satisfy a predetermined condition. For example, if a player has a type A card and a type B card in their hand, placing these two cards on the table activates a predetermined game effect, such as an attack spell, allowing them to acquire one card of the specified type on the table, for a total of three cards. Alternatively, in such a case, the player may leave the two cards in their hand on the table and acquire only the selected card on the table. The second combination condition is not limited to the above-mentioned "hand" and may be any condition that defines a specific combination. The first combination condition and the second combination condition may be the same combination condition. The combination condition may also be different for each player. Each combination condition is not limited to the combination of cards located in a specific area, such as the hand and the tableau cards, but may be the combination of cards located in any predefined area depending on the game, for example, it may be a condition related to the combination within the same area. It may also be a condition related to the combination with cards located in any of multiple areas.

[0134] In another embodiment, in addition to the guide image as described above, a frame image as described above may be projected onto each card placed on the play mat 10. This makes it easier to see the position of each card on the play mat 10.

[0135] Furthermore, the camera 27 may be a normal digital camera or, depending on the game content, an infrared camera, for example. Regarding the range photographed by the camera 27 and the installation position of the camera, the imaging range may be the same as the size of the play mat, or the imaging range may be greater than the size of the play mat 10, or the imaging range may be less than the size of the play mat 10. The camera 27 may be installed in any position that satisfies these conditions.

[0136] Furthermore, the above example shows an example of using an item detection AI to identify the information of each card in a camera image. That is, the example shows a method of identifying the type of card by querying a database for the identified content. In other embodiments, for example, characters or an identification code may be printed on game items such as cards. Then, by identifying this, the type of game item may be directly identified without querying a database.

[0137] Similarly to the camera 27, the projection range of the PJ 28 may be the same as the size of the play mat 10, or may be larger or smaller than the size of the play mat. The installation position of the PJ 28 may be any position as long as it satisfies these conditions.

[0138] In the above embodiment, the above-described processing is executed by a single PC 2. The PC 2 may include multiple storage devices and processors. The above-described processing may be executed by sharing the processing among these devices. The above-described processing may also be executed in a distributed system consisting of multiple information processing devices including a server. [Explanation of symbols]

[0139] 2. Information processing equipment 10 playmats 11 Game items 21 processors 22 Memory section 27 Camera 28 projectors

Claims

1. A game progress assistance system that assists a game that progresses using game items placed on a field, a camera that captures an image of the field; a projector that projects a projection image onto the field; an information processing device; The information processing device includes: an object information acquisition means for acquiring information about the object, including at least position information and type information about the object, based on an object image corresponding to the game item included in the captured image taken by the camera; a first combination determination means for determining, based on the information on the objects, a combination of any one of first objects located in a first area within the captured image and any one of second objects located in a second area within the captured image that satisfies a predetermined first combination condition; a projection image generating means for generating the projection image including a first combination display showing the combination that satisfies the first combination condition, Game progress support system.

2. The information processing device includes: a detection means for detecting, based on the position information, that each of the objects included in the combination of the first object and the second object determined to satisfy the first combination condition has been moved to a third area within the captured image; the projection image generating means generates a projection image including a third area display relating to the third area when it is detected that the object has been moved to the third area. The game progress assistance system according to claim 1 .

3. The game is a multi-player game, the first region is associated with a first player; The combination determination means determining, based on information about the objects, a combination of any one of the fourth objects located in a fourth area of ​​the captured image, which is an area associated with a second player, and any one of the second objects located in the second area, which combination satisfies the first combination condition; The projection image generating means a first combination display that indicates a combination of any one of the first objects located in the first area and any one of the second objects located in the second area that satisfies the first combination condition; a first combination display that indicates a combination of any one of the fourth objects located in the fourth area and any one of the second objects located in the second area that satisfies the first combination condition; generating the projection image comprising: The game progress assistance system according to claim 1 .

4. the first area and the third area are associated with a first player; the combination determination means determines, based on the information on the objects, a combination of any one of the fourth objects located in a fourth area of ​​the captured image, which is an area associated with a second player, and any one of the second objects located in the second area, which combination satisfies the first combination condition; the detection means detects, based on the position information, that each of the objects included in the combination of the second object and the fourth object determined to satisfy the first combination condition has been moved to a fifth area in the captured image, which is an area associated with the second player; the projection image generating means generates the projection image including a fifth area display relating to the fifth area when it is detected that the game item has been moved to the fifth area. The game progress assistance system according to claim 2 .

5. The game is a game in which multiple players compete against each other, taking turns; The projection image generating means a first generation mode in which the projection image is generated by including in the projection image a 1P-side combination display which is a first combination display indicating a combination of the first object and the second object that satisfies the first combination condition among the combinations of the first object and the second object, and not including in the projection image a 2P-side combination display which is a first combination display indicating a combination of the fourth object and the second object that satisfies the first combination condition among the combinations of the fourth object and the second object, and a second generation mode in which the projection image is generated by not including in the projection image the 1P-side combination display, but including the 2P-side combination display in the projection image, when movement of the object from the second area to a third area is detected, switching the generation mode from a first generation mode to a second generation mode; 5. The game progress assistance system according to claim 4, wherein when movement of the object from the second area to the fifth area is detected, the generation mode is switched from the second generation mode to the first generation mode.

6. The captured image includes: a second combination condition area including a presentation of a second combination condition regarding a combination of a plurality of types of the objects; generating a projection image including a second combination display in which each of the third objects arranged in the third area is associated with a presentation of the second combination condition associated with the third object among the presentations of the second combination conditions included in the second combination condition display area; The game progress assistance system according to claim 2 .

7. The information processing device includes: a second condition pattern information acquisition means for acquiring position information of the second condition pattern in the captured image based on an image of a second condition pattern that is included in the pattern of the field and that presents the second combination condition; and a field information acquiring means for acquiring position information of the field in the captured image based on position information of the second condition pattern, The projection image generating means generating the projection image based on position information of the field; The game progress assistance system according to claim 6.

8. The projection image generating means determining a layout position of the first combination display in the projection image based on the position information of each of the objects included in the combination that satisfies the first combination condition, and generating the projection image; 3. The game progress assistance system according to claim 1.

9. the first combination display includes a line connecting each of the frame displays corresponding to positions where each of the objects satisfying the first combination condition is arranged; The game progress assistance system according to claim 8.

10. the game items corresponding to the first objects are arranged in a line along a predetermined direction in an area on the field corresponding to the first area; the game items corresponding to the second objects are arranged in at least two rows along a predetermined direction in an area on the field corresponding to the second area; The projection image generating means 9. The game progression assistance system according to claim 8, wherein the first combination display is generated to include a frame display surrounding each of the objects that satisfy the first combination condition, and a line having a start point at a predetermined position on the outer edge of the frame display surrounding a first object located in the first area, the second area side, and an end point at a position within the frame display surrounding the second object, among combinations of objects that satisfy the first combination condition.

11. the frame display is a rectangular frame display, the line is a straight line having a start point at a predetermined position on the side of the frame display on the second area side and an end point at a center position of the frame display surrounding the second object; The game progress assistance system according to claim 10.

12. The game items are card-type game items each having a different picture on at least one side, the object information acquisition means acquires the type information based on the design of the game item; 3. The game progress assistance system according to claim 1.

13. A game progress assistance program executed by a processor of a game progress assistance system that assists a game progressed using a camera that captures an image of a field on which game items are placed, a projector that projects a projection image onto the field, and the game items placed on the field, The processor, an object information acquisition means for acquiring information about the object, including at least position information and type information about the object, based on an object image corresponding to the game item included in the captured image taken by the camera; a first combination determination means for determining, based on the information on the objects, a combination of any one of first objects located in a first area within the captured image and any one of second objects located in a second area within the captured image that satisfies a predetermined first combination condition; and causing the projection image generating unit to generate the projection image including a first combination display that indicates the combination that satisfies the first combination condition. Game progress support program.

14. The processor, functioning as a detection unit that detects that each of the objects included in the combination of the first object and the second object that is determined to satisfy the first combination condition based on the position information has been moved to a third area within the captured image; the projection image generating means generates a projection image including a third area display relating to the third area when it is detected that the object has been moved to the third area. The game progress assistance program according to claim 13.

15. The game is a multi-player game, the first region is associated with a first player; The combination determination means determining, based on information about the objects, a combination of any one of the fourth objects located in a fourth area of ​​the captured image, which is an area associated with a second player, and any one of the second objects located in the second area, which combination satisfies the first combination condition; The projection image generating means a first combination display that indicates a combination of any one of the first objects located in the first area and any one of the second objects located in the second area that satisfies the first combination condition; a first combination display indicating a combination of any one of the fourth objects located in the fourth area and any one of the second objects located in the second area that satisfies the first combination condition; The game progress assistance program according to claim 13.

16. the first area and the third area are associated with a first player; the combination determination means determines, based on the information on the objects, a combination of any one of the fourth objects located in a fourth area of ​​the captured image, which is an area associated with a second player, and any one of the second objects located in the second area, which combination satisfies the first combination condition; the detection means detects, based on the position information, that each of the objects included in the combination of the second object and the fourth object determined to satisfy the first combination condition has been moved to a fifth area in the captured image, which is an area associated with the second player; the projection image generating means generates the projection image including a fifth area display relating to the fifth area when it is detected that the game item has been moved to the fifth area. The game progress assistance program according to claim 14.

17. The game is a game in which multiple players compete against each other, taking turns; The projection image generating means a first generation mode in which the projection image is generated by including in the projection image a 1P-side combination display which is a first combination display indicating a combination of the first object and the second object that satisfies the first combination condition among the combinations of the first object and the second object, and not including in the projection image a 2P-side combination display which is a first combination display indicating a combination of the fourth object and the second object that satisfies the first combination condition among the combinations of the fourth object and the second object, and a second generation mode in which the projection image is generated by not including in the projection image the 1P-side combination display, but including the 2P-side combination display in the projection image, when movement of the object from the second area to a third area is detected, switching the generation mode from a first generation mode to a second generation mode; 17. The game progress assistance program according to claim 16, wherein when movement of the object from the second area to the fifth area is detected, the generation mode is switched from the second generation mode to the first generation mode.

18. The captured image includes: a second combination condition area including a presentation of a second combination condition regarding a combination of a plurality of types of the objects; generating a projection image including a second combination display in which each of the third objects arranged in the third area is associated with a presentation of the second combination condition associated with the third object among the presentations of the second combination conditions included in the second combination condition display area; The game progress assistance program according to claim 14.

19. The processor: a second condition pattern information acquisition means for acquiring position information of the second condition pattern in the captured image based on an image of a second condition pattern that is included in the pattern of the field and that presents the second combination condition; functioning as a field information acquisition means for acquiring position information of the field within the captured image based on position information of the second condition pattern; The projection image generating means generating the projection image based on position information of the field; The game progress assistance system according to claim 18.

20. The projection image generating means determining a layout position of the first combination display in the projection image based on the position information of each of the objects included in the combination that satisfies the first combination condition, and generating the projection image; 15. The game progress assistance program according to claim 13 or 14.

21. the first combination display includes a line connecting each of the frame displays corresponding to positions where each of the objects satisfying the first combination condition is arranged; 21. The game progress assistance program according to claim 20.

22. the game items corresponding to the first objects are arranged in a line along a predetermined direction in an area on the field corresponding to the first area; the game items corresponding to the second objects are arranged in at least two rows along a predetermined direction in an area on the field corresponding to the second area; The projection image generating means 21. The game progress assistance program according to claim 20, wherein the first combination display is generated to include a frame display surrounding each of the objects that satisfy the first combination condition, and a line having a starting point at a predetermined position on the outer edge of the frame display surrounding a first object located in the first area, the second area side, and an ending point at a position within the frame display surrounding the second object, among combinations of objects that satisfy the first combination condition.

23. the frame display is a rectangular frame display, the line is a straight line having a start point at a predetermined position on the side of the frame display on the second area side and an end point at a center position of the frame display surrounding the second object; 23. The game progress assistance program according to claim 22.

24. The game items are card-type game items each having a different picture on at least one side, the object information acquisition means acquires the type information based on the design of the game item; 15. The game progress assistance program according to claim 13 or 14.

25. A computer of a game progress assistance system that assists a game that progresses using game items placed on a field, acquiring information about the object, the information including at least position information and type information about the object, based on an object image corresponding to the game item included in an image captured by a camera that captures an image of the field; determining, based on the information on the objects, a combination of any one of first objects located in a first area within the captured image and any one of second objects located in a second area within the captured image that satisfies a predetermined first combination condition; generating a projection image including a first combination display showing the combination that satisfies the first combination condition; causing a projector that projects a projection image onto the field to output the generated projection image; How to help progress in the game.

26. a field in which game items are placed; a camera installed at a position capable of capturing the field within its imaging range; a projector installed in a position capable of projecting a projection image onto the field; an information processing device; The information processing device includes: an object information acquisition means for acquiring information about the object, including at least position information and type information about the object, based on an object image corresponding to the game item included in the captured image taken by the camera; a first combination determination means for determining, based on the information on the objects, a combination of any one of first objects located in a first area within the captured image and any one of second objects located in a second area within the captured image that satisfies a predetermined first combination condition; a projection image generating means for generating the projection image including a first combination display showing the combination that satisfies the first combination condition, Entertainment facilities.

Images