Virtual space display device and program

Abstract

To provide, to a user, play sensation equivalent to real one, in a card game or in a board game in a virtual space.SOLUTION: A virtual object is assigned to such a real object that identification ID, a position and a posture can be detected, and in a virtual space, the virtual object assigned to the real object is displayed on a position in the virtual space corresponding to the position of the real object, in a posture corresponding to the posture of the real object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] Virtual reality, hyper reality, and mixed reality. [Background technology]

[0002] In recent years, games that utilize augmented reality (AR), virtual reality (VR), and mixed reality (MR) have appeared (Patent Document 1). In these VR games, objects in the game are virtually represented, and the user progresses through the game by touching the virtual objects using a handheld pointing device or the like.

[0003] In addition, games that were previously played with real cards and pieces have been implemented as computer games, making game progression and card management easier for users than when dealing with real cards and pieces. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2020-064652 Summary of the Invention [Problem to be solved by the invention]

[0005] By playing card games or board games as computer games, users can automate the complicated game progress process and check various information such as scores in an easy-to-understand format. On the other hand, card games and board games are not only enjoyable due to the gameplay such as the rules of the game, but also because users can touch and handle the cards or game pieces, such as holding them in their hands or lining them up. These advantages are lost when playing them as computer games or VR games.

[0006] Therefore, the present invention provides a system that maintains the advantages of turning games that involve cards and pieces, such as card games and board games, into computer games, while also providing the fun of dealing with real cards and pieces. [Means for solving the problem]

[0007] A virtual object is assigned to a real object whose identification ID, position, and orientation can be detected, and the virtual object assigned to the identification ID of the real object in a virtual space is displayed at a position in the virtual space corresponding to the position of the real object, and in an orientation corresponding to the orientation of the real object.

[0008] The allocation of virtual objects to real objects is changed according to the progress of the game. Effect of the Invention

[0009] The present invention makes it possible to provide a system that maintains the advantages of turning games that involve cards and pieces, such as card games and board games, into computer games, while also providing the fun of dealing with real cards and pieces.

[0010] In the present invention, since the progress of the game can be managed by a computer, cheating can be prevented. Also, random elements such as shuffling of cards can be managed by a computer. For example, in the case of cards, it is possible to prevent the order of cards from being sufficiently random depending on the shuffling method, and certain cards from remaining together. Also, it is possible to intentionally control elements that are originally random according to the rules of the game. While having the above features, the present invention allows users to play the game using real cards and pieces. It is possible to obtain the convenience that can be obtained by making it a computer game, such as shuffling, card selection, automating game processing, and displaying various information such as the number of cards, while retaining the fun of users handling cards and pieces that is lost in computer games. [Brief description of the drawings]

[0011] [Figure 1] Actual card example [Diagram 2] Example of recognition of actual card [Diagram 3] Modification of the card face of the virtual card [Figure 4] System configuration example [Diagram 5] An example of the board DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Real cards and pieces are referred to as real objects. Cards and pieces that are virtually managed and represented on a computer, such as in a game, are referred to as virtual objects.

[0013] Below, we will mainly explain the application of the present invention to card games, but the present invention can also be applied to things other than cards, such as figures and pieces used in board games.

[0014] In this embodiment, the application to a card game in augmented reality (AR) will be described. In addition to augmented reality, the application to virtual reality (VR), mixed reality (MR), and projection mapping is also possible. The application to technology for overlaying images on real-world objects or displaying videos or images on the surface of real-world objects is also possible.

[0015] The present invention assigns virtual cards to multiple real cards with different identifiers. Also, the assignment of real cards and virtual cards is changed according to the progress of the game. This allows, for example, in a card game system using augmented reality, to automate operations such as shuffling and searching for a specific card, which are cumbersome for a user. For example, by shuffling the assignment of virtual cards to a set of real cards in a deck, the same effect as actually shuffling the deck can be obtained. In another example, when a user wants to select a specific card from the deck and add it to his / her hand, the user can assign the top card of the deck to that card, thereby saving the user the trouble of searching for the desired card among multiple cards as when using real cards.

[0016] The game system can perform processing on the assigned virtual card according to the position, orientation, etc. of the real card. In addition to using a camera to detect real objects such as cards and pieces, the game system can also incorporate a mechanism for detecting the position and orientation of the real object. For example, a barcode or a marker may be provided. In this case, the mechanism for identification and the mechanism for detecting the position and orientation may be separated. It is possible to combine an identification means that can identify a real object but has difficulty in detecting the exact position and orientation, with a mechanism that can detect the exact position and orientation of a real object but has difficulty in identifying it.

[0017] A virtual card assigned to a real card is drawn in the virtual space at a position in the virtual space corresponding to the position of the real card captured by a camera, etc. In an example of application to AR, the face of the virtual card assigned to the real card is drawn on the face of the real card in the image captured by the camera, allowing the user to see an image or video in which the real card has been replaced with the virtual card.

[0018] The allocation of real objects and virtual objects can be changed as the game progresses. Changing the allocation of real cards in a card game will be described below.

[0019] Explain how to shuffle cards. Changing the order of cards can be achieved by changing the allocation of actual cards, randomizing the order of some or all of the cards to be shuffled, or rearranging some or all of the cards into a predetermined order.

[0020] In a card game in which cards are drawn from a deck of cards whose order is unknown, an embodiment of shuffling the deck will be described. The shuffling process will be described in a case where virtual cards are assigned in advance to real cards in the deck, and in a case where virtual cards are assigned to cards to be drawn when cards are drawn from the deck.

[0021] An example of the case where cards are assigned in advance will be described. The order of the cards can be changed by rearranging the assignment of the physical cards in the deck. In this case, it is necessary to manage whether each physical card is included in the deck, i.e., the set of cards to be shuffled.

[0022] An example of assigning a virtual card to a card to be drawn will be described. A card may be taken out of a set such as a deck whose order is not disclosed, and the card information is made public; that is, when the top card of the deck, whose name is not known, is drawn and added to the hand, a virtual card may be assigned to the real card drawn. If the deck is randomly shuffled, a real card may be randomly assigned from the set of unassigned virtual cards. If the deck is manipulated in the course of the game and the next card to be drawn is already determined, the virtual card may be assigned to the real card drawn.

[0023] Explain card selection. A specific virtual object is selected from a collection of virtual objects, such as a deck of cards, and assigned to a real object. In a card game, for example, this is applied to an operation of selecting a specific card from the deck of cards and adding it to one's hand.

[0024] A case where a user selects a card from the deck and adds it to his / her hand will be described. When this situation occurs, the system presents a list of options to the user. For example, the presenting method is to display the cards of the options in a virtual space. The user selects the card by touching the card they want to select. At this time, a physical card may be used as a means for indicating the option. Since the system is equipped with a means for detecting the position of the physical card, it is possible to simplify the processing by having the user select the option using the physical card as a pointer rather than detecting the user's hand and determining the selected card from its position, since it is not necessary to detect the position of the user's hand. Also, using a means for detecting the position of the physical card often reduces false detections.

[0025] The selection may be made by displaying option cards on a display, or by using various input means such as buttons. Any selection means or method may be used depending on the game to which the selection is applied.

[0026] As an example of selection from a private collection in which the order of cards is not disclosed, a case of selecting cards from a deck or the like will be described. The top real card of the deck is assigned to the selected virtual card. This allows the user to take the selected card by taking the top card of the deck. At this time, the cards may be displayed so that the contents of the selected card can be seen. For example, the top card of the deck is the selected card and is in a face-up state. At this time, the contents of the card may not be seen by the opponent. For example, the contents of the card are not displayed on the opponent's display means. The user may take the top real card of the deck, use the real card like the wand to select a card, and assign the selected virtual card to the real card.

[0027] Regarding the state of the deck after the selected virtual card is assigned to the top real card of the deck, if a virtual card has already been assigned to the real card of the deck, the assignment of the virtual card to the top real card of the deck can be reproduced by replacing the real card to which the selected virtual card was assigned. After this, cards other than the selected card may be shuffled. The order of the cards after the selected card is assigned to the top of the deck may be freely set.

[0028] If the deck is unallocated, the selected card may be assigned to a card that is easy for the user to pick up, such as the top card of the deck. Even if the cards are stacked, the top card is visible and can be recognized by the system using a camera, so the system does not need to know the order of all the cards in the deck. If there are multiple selected cards, they can be handled by processing them one by one.

[0029] Also, an unassigned real card may be used like the aforementioned pointer wand to select a virtual card from the options, and the selected virtual card may be assigned to the real card used for card selection.

[0030] After the selection, any unassigned real cards picked up by the user may be detected and assigned the selected virtual card.

[0031] We will now explain the case where a card is selected from a set whose order is public. For example, a discard pile. As in the case of the deck, if the top card of the discard pile is changed to the selected card, the user can easily pick up the selected card. In cases where the order of cards is public, the order of cards may have meaning depending on the game to which the system is applied, so the system changes the allocation of virtual cards so that the order of cards after the selected card is removed is maintained or the order conforms to the rules of the game to which the system is applied. In this case, the system records the order in which the real cards are placed in the discard pile so that the order of the real cards in the discard pile can be recorded.

[0032] An example of assigning a selected virtual card to the topmost real card while maintaining the sorting order will be described below. First, the virtual card assigned to the topmost real card is temporarily stored, and the selected virtual card is assigned. Next, the temporarily stored virtual card is exchanged with the virtual card assigned to the real card one card below. Furthermore, the virtual card assigned to the real card one card below is exchanged with the temporarily stored virtual card. The above-mentioned exchange with the temporarily stored virtual card can be repeated until the selected virtual card becomes the real card that was assigned.

[0033] The selected virtual card may be assigned to an unassigned real card. In this case, the unassigned real card to which the virtual card is assigned may be selected from cards belonging to another set, such as a deck. In this case, the real card to which the selected card was assigned may be changed to an unassigned state.

[0034] The exchange of cards will now be explained. A case where a real card to which a virtual card has already been assigned is changed will be described. For example, a card in the hand is exchanged with a card on the table, or the position of the card is exchanged. In this case, the assignment of the original real card and the assignment of the destination real card can be exchanged. If the destination card is an unassigned virtual card, the virtual card assigned to the original real card is changed to an unassigned state, and the set to which it belongs is changed to the set to which the destination virtual card belonged. For example, when exchanging an assigned card in the hand with an unassigned card in the deck, the real card in the hand is left as it is, and the assignment of the virtual card of the real card is changed, and the originally assigned virtual card is changed to an unassigned card in the deck.

[0035] Now, the case of moving a card to another collection will be described. When moving a card from the tabletop or hand to a public set such as the discard pile, there is no need to perform an allocation change process. The user can simply place a real card from the tabletop or hand to the discard pile, so the system may display a message encouraging the user to do so. When moving an assigned card to an unassigned set, the target real card is put into an unassigned state, and the assigned virtual card is moved to the destination set.

[0036] When it is necessary to move a card to a distant location, such as during a game against an opponent in a remote location in a communication game, the real card to be moved is changed to an unallocated state, and the virtual card to be moved is assigned to the opponent's unallocated real card. The real card to be assigned may be an unallocated real card that is not currently being used, such as a card from the opponent's deck. For example, the top card of the deck is used. The user receiving the card may not use a real card, and the received card may be displayed and handled in the virtual space without being assigned to a real card.

[0037] We will now explain what happens if a user loses their physical card during play. A case where a user drops a real object that should be in hand and is no longer able to handle it will be described. When a virtual object is displayed in virtual reality, the user will often wear a virtual reality display means such as VR goggles. In such a state, it is difficult for the user to find or pick up a lost object. Therefore, when a real object is lost, the user can smoothly continue the game by allocating the virtual object that was assigned to the lost real object to another real object. When a lost real object is found during the game, the real object may be made available again as an unallocated real object. At this time, the system may instruct the user to place the real object in a specific location, such as the top of the deck.

[0038] Explain how the system detects the loss of a real object. If the system is unable to detect a real object for a predetermined period of time using the real object detection means, it determines that the real object has been lost, and assigns a virtual object to a new real object. Only specific real objects may be subject to loss detection, for example, only real cards on the table or in a hand may be subject to loss detection. Hidden real objects, such as discarded cards where cards are stacked and cannot be seen, may not be subject to loss detection.

[0039] When the system detects the loss of a real object, it may display that fact to the user and execute a process of allocating a new real object. At this time, if the system can recognize the real object whose loss was detected, it may cancel the allocation process.

[0040] The user may notify the system of the loss of an object through user action.

[0041] The lost virtual object may be treated as a virtual object without being assigned to a real object. The virtual object may be handled by a user's gesture or a user's input to an input means.

[0042] A method for handling a case where the number of virtual objects exceeds the number of real objects will be described. If the number of virtual objects to be assigned exceeds the number of real objects, a real object from a set other than the set of real objects to be assigned may be used. For example, when drawing a card from deck A, if there are no real cards from deck A, a virtual card from deck A may be assigned to a real card from another deck B instead and used. Alternatively, an unused virtual card from the discard pile may be assigned to an assigned real card, and the originally assigned virtual card from the discard pile may be treated as a virtual card. When the discard pile is used, the assignment process to the real card is performed again.

[0043] A virtual display using virtual reality or augmented reality may be used instead. When a user operates a virtual object that is being virtually displayed, a process of changing the allocation and allocating the virtual object to a real object may be performed.

[0044] This section explains matters related to the recognition of real objects such as cards and pieces. Here, the explanation will be mainly given using cards as an example, but the same can be applied to other three-dimensional objects such as pieces.

[0045] The position of the real object is detected by attaching markers (101, 102, 103, 104, 100) to the real object and detecting the identification ID, position, and orientation of the marker. The system uses the detected ID, position, and orientation information to display a virtual object associated with the real object on the virtual reality display means. In the case of a card, an image of a virtual card is displayed on the face of a real card captured by a camera, so that the real card appears to the user as a virtual card.

[0046] The arrangement of markers on real cards will be explained. When a user holds multiple cards in his / her hand, some cards will overlap. Also, some cards may be hidden by the hand holding them. By arranging multiple markers (101, 102, 103, 104) on one real object, the system can recognize the ID, position, and orientation of the real object even if some of the real object is hidden.

[0047] One example of marker placement is to place markers at least on the corners of the cards as shown in Figure 1. In the case of square-shaped cards, a typical hand holds cards as shown in Figure 2. In this case, at least one of the four corners of overlapping cards will be visible, so by placing markers as in the cards in Figure 2, it is possible to recognize cards that are partially hidden. In this case, different IDs can be used for the markers placed on one real object. The system changes the position at which the virtual card is displayed based on the ID information. For example, if a marker is detected in the upper left corner of a card, the marker is displayed so that its position matches the position corresponding to the marker in the upper left corner of the virtual card.

[0048] The detection of the position of the marker within one real object may use the recognition result of the real object itself. In the case of a card, for example, the edge of the card, i.e., a straight line, is detected, and from the positional relationship between the marker nearby and the edge of the card, it is detected which of the four corners of the real card the marker is placed at. For example, if the edge of the card is detected to the left and above the marker, the marker becomes the top left marker of the card. In such a case, it is possible to reduce the number of types of IDs of the markers placed on the real object. In other words, it is sufficient to prepare at least one type of ID for one real object, and which marker on the real object it is can be identified from the shape of the real object based on the recognized marker, so that the number of types of IDs used by the system can be reduced, and as a result, the marker can be made smaller, the pattern of the marker can be simplified, and the ID can be easily identified. In this case, the marker includes orientation information.

[0049] In addition to markers, barcodes (111, 112, 113, 114, 115, 116) can also be used. In this case, the markers can be used to recognize position information and detect the identification ID of the real object. In the case of cards, placing a barcode near the edge of the card increases the number of situations where the cards can be detected when overlapping. Also, like barcodes 113 and 114, multiple barcodes can be placed on one side, or one barcode can be placed on one side.

[0050] A large marker may be placed so that it is easy to recognize, such as the marker 100. In general, when the size of the marker or barcode is small, there is a risk of false detection. When the marker is large, it is often hidden and cannot be recognized, but in the case of card games, when the card is used as a tableau card, the entire surface of the card is often visible, so this is effective.

[0051] The descriptions regarding the markers and barcodes of these physical cards can also be applied to three-dimensional objects including curved surfaces. Furthermore, the markers and barcodes may be a single color or a combination of multiple colors. Furthermore, the markers, barcodes, and card surface 131 may use different colors or patterns for each ID of the physical card. In particular, by changing the color of the card surface for each ID, the card surfaces are separated by color when the physical cards are stacked, which is useful for detecting the card surface of each physical card.

[0052] In the case of cards, different marker and barcode layouts may be used on the front and back of the card. It is not necessary to provide a marker or barcode on the back. Also, the ID of the marker or barcode on the back may be different for the front and back, or the same ID may be used. It is not necessary to distinguish between the front and back.

[0053] This section describes the area in which virtual objects are rendered. Here, we will mainly use examples of cards, but application to three-dimensional objects and curved surfaces is also possible.

[0054] A case where the physical card in Fig. 1 is used will be described. The physical card in Fig. 1 has markers at the top right 101, bottom right 102, bottom left 103, and top left 104. The edge of the card may also be provided with a contour 132 that makes it easier to detect the edge of the card. The markers and contour 132 can use colors and patterns that are easy for a system such as a camera to recognize.

[0055] The system recognizes the position of the marker by means of a camera.

[0056] The system detects card surface 131. Card surface 131 may have a color, pattern, or other material features that make it easy to detect. Alternatively, contour 132 may be detected, and the area surrounded by contour 132 and including the marker may be recognized as card surface 131. In the case of FIG. 2, contour 132 can be taken as the boundary between the card surfaces of the cards. The marker portion and contour portion may be treated as part of the card surface.

[0057] When an image of a real card recorded in the system is placed at a position and in a posture corresponding to the marker captured by camera 402, a portion of the image captured by camera 402 that matches the placed real card may be detected as the card face. Here, the portion of the card face visible to camera 402 matches the image of the placed real card, and the hidden portion does not match, so that the system can detect the area in which the card face of the virtual card should be drawn. A pattern matching process may be performed to detect the matching portion. The acceptable degree of match may be adjusted as appropriate.

[0058] The system deforms the shape of the image of the card face of the virtual card corresponding to the marker according to the detected marker's posture. An example is shown in Figure 3. The system applies a deformation (134) to the image of the card face, which changes the magnification ratio of the detected marker 106 relative to the size of the recorded marker 105, and changes the image of the card face in the same way as the change in posture of the marker.

[0059] The system uses the detected card surface 131 as a mask and ANDs it with the card surface of a virtual card that has been deformed according to the magnification ratio and posture of the detected marker, and cuts out only the part of the card surface of the virtual card that corresponds to the part visible to the camera.

[0060] The system draws an image of the deformed card face. In the case of AR or MR, the deformed image 134 after the cutout process is drawn in a location corresponding to the position of the detected marker, so that the actual card in the image captured by the camera is displayed as a virtual card. In cases where it is not necessary to display real space, such as in VR, it is not necessary to perform the process of detecting the portion of the card face 131 of the virtual card that is visible to the camera or the process of cutting out the card face of the virtual card.

[0061] The order of processing is not limited to the above-mentioned order. For example, the order of image deformation of the card face and detection of card face 131 may be reversed. In this case, the detected card face is deformed so that it appears as if card face 131 were captured in a specified posture, such as from the front, and the image of the virtual card is cropped with a mask corresponding to its shape, thereby extracting the visible portion of the card face that the system should draw.

[0062] The drawing of a virtual object will now be described. Here, we will mainly explain the drawing of virtual cards in augmented reality (AR).

[0063] A virtual object is displayed in a virtual space according to the position and orientation of the real object. By displaying a virtual object in a video or image in the real space, the real object can be made to appear as a virtual object.

[0064] The displayed virtual objects do not have to be based on real objects. For example, in the case of a card game, instead of simply displaying cards, it is also possible to display CG images of characters corresponding to the cards. The characters may have animations and may move in accordance with the progress of the game.

[0065] Also, information about the displayed virtual object may be displayed. For example, the status of the card or information about other cards that affect or are affected by the card may be displayed. Examples of the representation include displaying a frame for displaying information near the virtual object, connecting the displayed virtual objects with arrows or lines, etc. Animation may be added to the image displayed on the card surface of the virtual object, or information may be displayed.

[0066] The displayed virtual object may be made transparent. For example, when a user is about to touch a real object, the displayed virtual object may be made transparent, allowing the user to grasp the shape of the real object, making it easier for the user to handle the real card. This is particularly useful when the shape of the real object and the shape of the displayed virtual object are different.

[0067] In addition to making the virtual object transparent, the shape of the virtual object may be changed to a shape similar to that of the real object. In this case, the displayed virtual object may be displayed transparently and overlaid. In addition to making the shape of the virtual object the same as the real object, only a part of the virtual object, such as a handle provided on the real object, may be changed to a shape similar to that of the real object.

[0068] The displayed virtual object may reflect the state of the real object. For example, in a case where a real card has a shadow, the shadow can be displayed on the displayed virtual card to enhance the affinity of the representation with the real world. One example of a method for acquiring the state of a real object is to obtain the difference between the information of the real object recorded in the system and the state of the real object recognized by the system. For example, if the recognized real object is recognized as being darker than the recorded real object, the color of the displayed virtual object is darkened.

[0069] Next, changing the orientation of a virtual object will be described. When displaying a virtual object, the virtual object may be displayed regardless of the orientation or position of the real object. In the case of a card, the orientation or front or back of the card may be changed. For example, when applied to a card game, when the user needs to change the orientation or front or back of a card according to the rules of the game, the orientation or front or back of the virtual card displayed by the system on behalf of the user may be changed. Also, in a card game, when the front of a card that is not normally visible to the opponent, such as a hand card, is visible to the opponent, the opponent's display means may process and display the card so that the information on the front side is not visible. One example is to display a specific image on the card face, or to display the back side.

[0070] In one example, instead of turning over a real card, a user can make a gesture such as touching the real card to reveal the face-up side of a corresponding virtual card, or when the rules of the applicable game require a card to be turned face-up, the system may automatically reveal the face-up side of the card.

[0071] The size of the displayed virtual object may be changed to a size independent of the real object. For example, the size of the displayed virtual card is made larger than the size of the actual card.

[0072] When changing the display of a virtual object, animation may be used. By changing the display of a virtual object with animation, the user can intuitively grasp the change in the virtual object.

[0073] In the present invention, since the progress of the game can be managed by a computer, cheating can be prevented. Also, random elements such as shuffling of cards can be managed by a computer. For example, in the case of cards, it is possible to prevent the order of cards from being sufficiently random depending on the shuffling method, and certain cards from remaining together. Also, it is possible to intentionally control elements that are originally random according to the rules of the game. While having the above features, the present invention allows users to play the game using real cards and pieces. It is possible to obtain the convenience that can be obtained by making it a computer game, such as shuffling, card selection, automating game processing, and displaying various information such as the number of cards, while retaining the fun of users handling cards and pieces that is lost in computer games.

[0074] It will also be possible to play against other players via the Internet. EXAMPLES

[0075] An example of the configuration is shown below. Control / program execution is performed by a combination of general computers, microcomputers, and circuits. FIG. 4 shows an example of the system configuration when applied to a card game. The system includes real cards, a computer 401, a camera 402 as real object recognition means, and a board 403. The system also includes display means for displaying virtual cards to the user. An example of the display means is a head mounted display (HMD) 405, and the user can view a virtual space via the HMD 405, or an image observed by a camera provided in the HMD 405, in which the virtual cards are displayed. Here, the camera provided in the HDM 405 may be the camera 402 for recognizing real cards. A plurality of cameras 402 may be used, and the plurality of cameras may be used to complement each other's blind spots and improve the recognition accuracy of position and orientation. The computer 401 is equipped with a game program 404, and the progress of the game is managed by the game program 404. The computer 401 also recognizes the type, position, and posture of real cards based on images from a camera 402, and displays virtual cards assigned by the game program 404 on an HMD 405, which is a display means.

[0076] The board 403 is used for placing cards, a deck, discards, and the like in the game to which the system is applied. FIG. 5 shows an example of the board. The board 403 in this example includes a deck area 501, a discard area 502, a deck area 503, and a second deck area 504. Each area may be provided with a marker for identifying the location of the area. The marker of the area is hidden when a card is placed thereon, but by recording the location of the area when no card is placed thereon, the system can determine whether the card at that location is a deck, a discard, or a deck. If the initial state of the board is determined according to the rules of the game, the board 403 and the markers on the board 403 are not provided, and the initial arrangement of the actual cards may be recorded to perform the above determination. The HDM 405 may display the location of the area so that the user can identify the location of each area (501, 502, 503, 504).

[0077] The real object or the board may be provided with a means for detecting user input, such as a contact sensor. For example, a touch sensor is provided in the place 503 for placing cards on the board 403, and when a user inputs to the touch sensor, a predetermined action in the game to which the action is applied is executed according to the virtual card assigned to the real card placed on the place. One example is executing an attack in the game or a process specific to the real card, or displaying candidate actions so that the user can select and execute the action.

[0078] When there is an opponent, the system is provided with a board 403 for the opponent and an HMD 405. Depending on the game to which the system is applied, one board 403 may be shared by all users.

[0079] The opponent may be a user at a remote location connected by a network or the Internet. In that case, the user and the opponent each use the system shown in FIG. 4. The computer 401 is connected by a network to play a communication game. In this case, the opponent's board 403 and virtual objects may not be provided, and the objects handled by the opponent, such as cards, may only be displayed virtually. Also, accurate position and posture information may not be acquired for the opponent's real objects. For example, the positions of the real cards that move according to the opponent's user's movements do not need to be displayed accurately for the opponent's hand, and they can be displayed at a predetermined location in the virtual space, for example. When the opponent puts a real card from his hand onto the tableau, the movement of the real card does not need to be reproduced, and the virtual card may be moved by a predetermined animation or the like. The movement of the opponent's virtual card may be the same as when the opponent is a computer player of a general computer game.

[0080] User input such as the opponent's movement or voice may be converted into a predetermined action, a predetermined line, or a predetermined voice of a character in the virtual space. The conversion may be performed in the user's system or the opponent's system. It may also be performed in a system at a relay point of communication. For example, when an opponent plays a card on the table, the character representing the opponent performs a predetermined action of playing a card on the table. In addition, at this time, the line "playing a card on the table" may be accompanied. Since the user's movement and voice are replaced with the character's predetermined movement and line, etc., and transmitted to the opponent, this has the effect of increasing the anonymity of the user from the communication opponent. In addition, since the opponent is replaced with a game character, etc., the user can enjoy a human-to-human battle while maintaining the worldview of the game.

[0081] Each component may be replaced with another component having similar effects or characteristics.

[0082] Instead of using the camera 401, other means for recognizing the ID, position, and orientation of a real object may be used. The real object itself may be provided with a means for recognizing its own position. One example is a GPS. The real object may be provided with a means for recognizing a marker or a barcode on the board surface 403.

[0083] The system of the present invention may be implemented as an application for a smartphone. In this case, the camera 402 may be a camera provided on the smartphone. The smartphone may also function as the HMD 405. A virtual reality or augmented reality image may be rendered on the smartphone screen.

[0084] The function of one component in the present embodiment may be separated into multiple components, or the functions of multiple components may be integrated into one component. At least a part of the configuration of the present embodiment may be replaced with a known configuration having a similar function. A part of the configuration of the present embodiment may be omitted. At least a part of the configuration of the present embodiment may be added to or substituted for another configuration of the present embodiment. All aspects included in the technical idea identified only by the wording described in the claims are embodiments of the present invention.

[0085] The present invention is not limited to the above-described embodiment, and various modifications can be made. In addition to the above-mentioned device, the present invention can also be realized in various forms, such as a system that includes the device as a component, a program for causing a computer to function as the device, a medium on which this program is recorded, a control method, a method, etc.

[0086] In addition, the program may be installed on a server and downloaded by the device to be executed from the server, or a part or all of the processing may be executed on the server side. [Industrial Applicability]

[0087] The technology of the present application is applicable to various board games and card games. Furthermore, even without managing the allocation of virtual objects to real objects, the user can enjoy multiple types of games using only the system of the present application by simply changing the type of virtual object to be assigned depending on the game the user is playing. For example, in a card game, the user can enjoy a game using playing cards and Hanafuda cards by switching between playing cards and Hanafuda cards to be assigned to virtual cards. Furthermore, in a card game in which the user builds a deck (mountain of cards), the user can virtually manage and build the deck and select which deck to use at the start of the game to play. [Explanation of symbols]

[0088] 10 User's hand, 100 Marker, 101 Marker top right, 102 Marker bottom right, 103 Marker bottom left, 104 Marker top left, 105 Recorded marker, 106 Detected marker, 111 Barcode top right, 112 Barcode bottom right, 113 Barcode bottom left, 114 Barcode top left, 115 Barcode bottom, 116 Barcode top, 131 Card surface, 132 Contour, 133 Card surface of virtual card, 134 Card surface of virtual card after transformation, 401 Computer, 402 Camera, 403 Board, 404 Game program, 405 Head mounted display, 501 Deck placement area, 502 Discard placement area, 503 Playing card placement area, 504 Deck 2 placement area

Claims

1. A real object is an object in the real world; an object recognition means for recognizing an identification ID, a position and an orientation of the real object; A virtual space display means; A virtual space representation device comprising: assigning a virtual object to be displayed in a virtual space to the identification ID; The virtual space display means displays the virtual object assigned to the identification ID of the real object recognized by the object recognition means. displaying the recognized real object in the virtual space at a position and orientation corresponding to the recognized position and orientation of the real object; A virtual space rendering device in which the assignment of the virtual object to the identification ID is changeable.

2. A game device comprising the virtual space representation device of claim 1, The allocation of virtual objects to real objects is changed according to the progress of the game. Gaming device.

Images