Virtual Space System

Abstract

To provide a virtual space system that has improved user operability while maintaining, to some extent, the effect of being able to provide realistic sensations to a user.SOLUTION: A virtual space system includes a handy type client device having a touch panel display operated by a user. The client device has means for displaying part of an image present in a virtual space on the touch panel display, means for accepting a swipe operation performed by the user, and means for causing a display area of the image to transition according to a swipe direction of the swipe operation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0005] , , , ,

[0004]

[0001] The present invention relates to a virtual space system, and more particularly to a virtual space system that can be displayed using a web browser.

Background Art

[0002] Patent Document 1 discloses a communication system by the group company (Space Lab Co., Ltd.) of the present applicant that does not use an application program. This communication system moves an avatar image imitating a user in a virtual space typified by a virtual exhibition hall, and communicates with another user through the avatar image imitating the other user at the destination.

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0003] According to the disclosure of Patent Document 1, since the avatar image can be moved relatively freely in the virtual space, there is an effect that a sense of presence as if the user is in the real space can be provided to the user. This type of communication system can be applied not only to virtual exhibition halls but also to virtual malls, virtual art museums, and the like.

[0004] However, when the operation for moving the avatar image is performed through a handy-type information processing device having a touch panel display such as a smartphone, it may not be preferable in terms of operability that the avatar image can be moved relatively freely in the virtual space.

[0005] Therefore, an object of the present invention is to provide a virtual space system that improves the operability of a user while maintaining to some extent the effect of providing a sense of presence to the user. [Means for solving the problem]

[0006] To solve the above problems, the present invention provides A virtual space system comprising a handheld client device having a touch panel display operated by a user, The aforementioned client device The means for displaying a portion of an image in a virtual space on the aforementioned touch panel display, A means for receiving a swipe operation performed by the user, Means for transitioning the display area of ​​the image according to the swipe direction of the swipe operation, It holds.

[0007] The transition of the aforementioned display area should preferably be performed in a manner that allows a virtual user to move along a virtual pathway.

[0008] Furthermore, a storage means for linking and storing virtual booths within the virtual space with the contact information of the person in charge of virtual user support in said virtual booth, A detection means for detecting when a virtual user in the virtual space has entered the virtual booth, A notification means that, when the arrival of the virtual booth is detected by the detection means, notifies the contact person corresponding to the virtual booth stored by the storage means, It can also be equipped with this feature.

[0009] Furthermore, the virtual space may be displayed using a web browser.

[0010] A server device connected to the aforementioned client device via a network, It can also be equipped with this feature. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described below with reference to the drawings.

[0012] Figure 1 is a block diagram showing a schematic configuration of a communication system to which the virtual space system of an embodiment of the present invention is applied. Figure 1 shows a server device 100, the first to nth client devices 200A to 200N (these will be referred to as "client device 200" without distinction), and a network 300, which will be described below.

[0013] The server device 100 is equipped with the function of providing web services. In response to initial processing requests from the client device 200, it transmits content images such as background images of virtual art museums and image files of artworks, as well as incoming notification display images, to be displayed in the virtual space. In response to running processing requests from the client device 200, it transmits control signals for displaying content images, video signals, and incoming notification display images in the virtual space, as well as control signals for outputting audio signals. This enables communication in the virtual space between one user operating the client device 200 and other users or personnel in charge of virtual user support.

[0014] The client device 200 is operated by the person in charge and the user. Each client device 200 has a web browser installed and generates video and audio signals based on control signals transmitted from, for example, the server device 100, and displays them on the display 510 (Figure 3) and outputs them from the speaker 520 (Figure 3) via the web browser. As the client device 200, information processing devices with communication functions, display functions, and audio input / output functions can be used, such as personal computers, tablet terminals, smartphones, and PDAs (Personal Digital Assistants). In this embodiment, the following explanation will mainly describe the case where a smartphone is used as the client device 200.

[0015] Network 300 is a general term for all networks that require the Internet. Network 300 can include networks such as local area networks (LANs) built downstream of the Internet, as well as mobile phone lines.

[0016] Figure 2 is a block diagram showing a schematic configuration of the server device 100 shown in Figure 1. Figure 2 shows the following components: video signal control means 110, audio signal control means 120, transmission amount control means 130, content storage means 140, calculation means 150, database 160, storage means 170, detection means 180, and notification means 190.

[0017] The video signal control means 110 generates control signals necessary to display the images shown in Figures 4 to 6, which will be described later. Specifically, the video signal control means 110 generates control signals to superimpose, for example, images of artworks or other items, and pop-up images as exemplified in Figure 6, onto a background image of a virtual museum or the like in a virtual space, using sprite processing or the like.

[0018] The audio signal control means 120 receives an audio signal based on the user's voice collected by a microphone 430 (Figure 3) attached to the first client device 200A, for example, and transmits a control signal to a predetermined client device 200 among the second to nth client devices 200B to 200N to output the corresponding audio signal.

[0019] The transmission volume control means 130 typically performs control, such as thinning out voice signals, to ensure that the total communication traffic of the network 300 does not exceed a predetermined threshold. The transmission volume control means 130 generates a control signal based on the communication traffic.

[0020] The content storage means 140 stores content displayed on the display 510 (Figure 3), such as content images, incoming notification display images, and each user's avatar images, in a storage medium (not shown).

[0021] The calculation means 150 calculates, for each client device 200 connected to the server device 100 via the network 300, the position information (coordinate information) of each virtual user in the virtual space, etc.

[0022] The database 160 stores the position information of each virtual user in the virtual space calculated by the calculation means 150. In addition, the database 160 also stores the position information (coordinate information) of each virtual booth by the producer of the virtual space, etc. These position information are used when generating control signals in the video signal control means 110, the audio signal control means 120, and the transmission amount control means 130.

[0023] Here, the virtual booth can be, for example, a virtual store in the case of a virtual mall, a virtual exhibition space for virtual artworks in the case of a virtual art museum, a virtual cage where virtual animals are located in the case of a virtual zoo, a virtual desk space where a virtual desk of a virtual user is placed in the case of a virtual office, etc.

[0024] The storage means 170 associates the virtual booth in the virtual space with the contact information of the person in charge of the virtual user corresponding to the virtual booth and stores it in a storage medium (not shown). It is preferable to use a unique ID assigned to the person in charge for the contact information here for using the communication system of this embodiment.

[0025] As other contact information, an account of a social network service (SNS) uniquely assigned to the person in charge, a phone number such as the IP phone of the person in charge, the email address of the person in charge, etc. can also be used. Also, when there are multiple persons in charge of virtual user correspondence for one virtual booth, the virtual booth and the multiple contact information may be associated and stored.

[0026] The detection means 180 detects when a virtual user in the virtual space enters a virtual booth, based on the location information of each virtual user and the location information of each virtual booth calculated by the calculation means 150.

[0027] The notification means 190, when the arrival of the virtual booth is detected by the detection means 180, notifies the contacts corresponding to the virtual booth stored by the storage means 170. If there are multiple such contacts, notifications may be sent to, for example, all of the online contacts.

[0028] The form of this notification is not specified, but for example, the speaker 520 (Figure 3) of the client device 200 operated by the person in charge can continuously output an incoming notification tone until the person in charge responds. In addition to this, or instead, the display 510 (Figure 3) of the client device 200 can continuously display an incoming notification.

[0029] Furthermore, if notifications have been sent to multiple contacts, if a response is received from any of the contacts, further notifications may be sent to the remaining contacts indicating that a response has been received. These further notifications may be sent, for example, by displaying them on the display 510. These further notifications may also include ending the continuous output of the ringtone and / or the incoming call display that was previously being sent.

[0030] Furthermore, some or all of the functions of some of the means provided in the server device 100 may be provided in the client device 200. Conversely, some or all of the functions of some of the means provided in the client device 200 may be provided in the server device 100. However, the notification means 190 is essential to be provided in the server device 100 due to its function.

[0031] Figure 3 is a block diagram showing a schematic configuration of the client device 200 shown in Figure 1. Figure 3 shows the video signal processing means 210, the audio signal processing means 220, the receiving means 230, and the transmitting means 240, which will be described below. Figure 3 also shows the input device 400 and the output device 500 attached to the client device 200.

[0032] Furthermore, when using a smartphone as the client device 200, or other handheld information processing devices such as PDAs, they typically have a touch panel display. Needless to say, in these cases, the touch panel display also serves as part of the input device 400 and output device 500.

[0033] The video signal processing means 210 receives control signals transmitted from the server device 100 via the network 300 in response to signals corresponding to user actions transmitted via the network 300 by the transmission means 240, generates a video signal based on the control signals, and outputs it to the display 510. The video signal processing means 210 also outputs a video signal of an incoming notification display image to the display 510 in accordance with a control signal from the notification means 190 of the server device 100.

[0034] The audio signal processing means 220 receives a control signal transmitted from the server device 100 via the network 300 based on the audio signal of another user transmitted from another client device 200 via the network 300, generates an audio signal based on the control signal, and outputs it to the speaker 520. The audio signal processing means 220 also outputs an audio signal related to the notification ringtone to the speaker 520 in accordance with the control signal from the notification means 190 of the server device 100.

[0035] The reception means 230 receives actions performed by the user of the client device 200 through the input device 400. Specifically, for example, it receives actions such as when the user performs a tap and / or swipe operation on the touch panel display, presses a cursor key on the keyboard 410, operates the mouse 420 (pointer movement, click operation, etc.), or makes a sound that is picked up by the microphone 430.

[0036] The transmitting means 240 transmits a signal corresponding to the user action received by the receiving means 230 to the server device 100 via the network 300. For example, if the action is a speech sound picked up by the microphone 430, an audio signal based on that speech is transmitted to the server device 100. Further details regarding the audio signal will be described later.

[0037] Furthermore, each user may be assigned a password along with their respective ID or login ID, and only users authenticated by these credentials may be able to receive services through the communication system of this embodiment.

[0038] In this embodiment, input devices 400 such as a keyboard 410, mouse 420, and microphone 430, and output devices 500 such as a display 510 and speaker 520 are used to enable each user of the client device 200 to communicate with each other in the virtual space.

[0039] Figures 4 to 6 show examples of images displayed on the display 510 of the first client device 200A shown in Figure 1. Figures 4 to 6 show examples of screens when the simulation system of this embodiment is applied to a virtual art museum. Although not shown in Figures 4 to 5, each user's avatar image may be superimposed on the background image and artwork images 600A to 600C of the virtual art museum.

[0040] Figure 4(a) shows an example image in which three painting images 600A to 600C and a switching display area 600D for switching the on / off state of the microphone attached to the first client device 200A are superimposed on the background image of a virtual museum.

[0041] These painting images 600A-600C can include images of paintings that exist as physical objects, or images of paintings that exist as digital data. They may also be images of paintings to which non-fungible tokens (NFTs) are linked.

[0042] Note that the image example in Figure 4(a) is intended to represent the state before a virtual user in the virtual space enters the virtual booth. Therefore, in this case, the arrival is not detected by the detection means 180, and the notification means 190 does not notify the person in charge of handling the virtual user related to the painting image 600B (for example, the commentator).

[0043] Figure 4(b) shows an example of the transition that occurs when the user swipes their finger downwards on the touch panel display of the first client device 200A, or when they tap above the center point of the screen, while the example image in Figure 4(a) is displayed. On the other hand, when the user swipes their finger upwards on the touch panel display of the first client device 200A, or when they tap below the center point of the screen, while the example image in Figure 4(b) is displayed, the image transitions to the example image in Figure 4(a).

[0044] The transition from the display state in Figure 4(a) to the display state in Figure 4(b) is executed when, for example, the user moves the pointer downwards while holding down the left button on the mouse 420, or when the user presses the up arrow key on the keyboard 410.

[0045] Furthermore, in the example image in Figure 4(b), in addition to the painting images 600A-600C and the switching display area 600D, a link display area 600E for displaying a webpage containing the following information is superimposed on the background image of the virtual museum and the painting images 600A-600C. The link display area 600E is associated with the URL assigned to that webpage.

[0046] The webpage contains, for example, a token name consisting of the artist's name and title of the painting related to the painting image 600B, a unique token ID assigned to the NFT, and the name of the owner of the painting. The link display area 600E may also be superimposed on the background image of the virtual museum in the example image in Figure 4(a).

[0047] The image example in Figure 4(b) is intended to represent the state after a virtual user in the virtual space has entered the virtual booth. In this case, the arrival is detected by the detection means 180, and the notification means 190 notifies the person in charge of handling the virtual user related to the painting image 600B.

[0048] Figure 4(c) shows an example screen that appears when the user taps the switching display area 600D with their finger or other object, or aligns the mouse pointer 420 with the switching display area 600D and clicks, while the example screen shown in Figure 4(b) is displayed. If the user taps the switching display area 600D again while the example screen shown in Figure 4(c) is displayed, the screen will revert to the example screen shown in Figure 4(b).

[0049] The on / off status of the microphone 430 shown in the switching display area 600D differs between the screen example in Figure 4(b) and the screen example in Figure 4(c). However, to make the status easier to see, the switching display area 600D can be displayed in different colors.

[0050] Figure 5(a) shows an example screen that appears when the user swipes their finger or other object touching the touch panel display of the first client device 200A to the left, or when they tap to the right of the center point of the screen, while the example screen shown in Figure 4(a) is displayed. Furthermore, when the user swipes their finger or other object touching the touch panel display of the first client device 200A to the right, or when they tap to the left of the center point of the screen, while the example screen shown in Figure 5(a) is displayed, the screen transitions to the example screen shown in Figure 4(a).

[0051] Furthermore, if, while the example screen shown in Figure 4(a) is displayed, the user performs a swipe operation by moving their finger or other object touching the touch panel display of the first client device 200A to the right, or taps to the left of the center point of the screen, the virtual user will transition to an example screen where they are positioned directly in front of the painting image 600A.

[0052] In this embodiment, a notable feature is that the operation to move a virtual user in the virtual corridor of the virtual art museum can be achieved simply by the user swiping their finger or the like that touching the touch panel display of the first client device 200A in the left-right direction (note that this does not mean that swiping in the up-down direction is excluded), thereby enhancing user operability.

[0053] In a virtual art museum, for example, there may be virtual walls that are perpendicular to each other, and virtual pathways in the vicinity of these walls may also be designed at almost right angles. However, by swiping left or right, the image processing makes it appear as if the user is moving along the virtual pathway, even through those right-angle sections.

[0054] With this type of image processing, if the first client device 200A is a handheld information processing device with a touch panel display, like a smartphone, then all the necessary processing in the virtual art museum can be performed by simply holding the smartphone in one hand and operating it with the thumb.

[0055] Figure 5(b) shows an example screen that appears when the user performs a swipe operation, such as moving their finger downwards, while the example screen shown in Figure 5(a) is displayed. Furthermore, when the user performs a swipe operation, such as moving their finger upwards, while the example screen shown in Figure 5(b) is displayed, the screen transitions to the example screen shown in Figure 5(a).

[0056] Figure 5(c) shows an example of the screen that appears when the user taps the switching display area 600D with their finger or other means while the example screen shown in Figure 5(b) is displayed. Furthermore, if the user taps the switching display area 600D again while the example screen shown in Figure 5(c) is displayed, the screen will revert to the example screen shown in Figure 5(b).

[0057] Here, in the communication system of this embodiment, as in Patent Document 1, control is performed to display the painting images 600A to 600C in the image example of Figure 4(b) larger than the image example of Figure 4(a).

[0058] The same control is applied to the example screens shown in Figure 5(a) and Figure 5(b). As a result, painting image 600C is displayed largely within the virtual museum, while painting image 600A is completely hidden, and part of painting image 600B is hidden within the display area.

[0059] Such display control is performed by the following process. First, when a user performs a swipe operation on the touch panel display of the first client device 200A, the receiving means 230 of the first client device 200A receives that action. Next, the transmitting means 240 of the first client device 200A transmits a signal corresponding to the action received by the receiving means 230 to the server device 100 via the network 300.

[0060] When the server device 100 receives the above signal transmitted from the first client device 200A, the calculation means 150 calculates the virtual user's position information (coordinate information) before movement or the position information (coordinate information) after movement relative to the origin (mutually orthogonal XYZ axis coordinates, X,Y,Z=(0,0,0)) based on the distance or speed and direction of movement of a finger or the like on the touch panel display, and stores it in the database 160.

[0061] Incidentally, for example, in the display state shown in Figure 4(a), if the user performs a swipe operation by moving their finger or other object that has touched the touch panel display of the client device 200 downwards, one method is to control the system so that it transitions to the display state shown in Figure 4(b), regardless of the distance or speed of movement of the finger or object.

[0062] Such transition control can be achieved by fixing the position information of the virtual user's stationary position within the virtual space, that is, by setting it to perform fixed-point movement. For example, if the position information of the virtual user in the virtual space in the display state of Figure 4(a) is represented as (x,y,z) in mutually orthogonal XYZ axis coordinates, then the position information of the virtual user in the virtual space in the display state of Figure 5(a) is set to (x+1,y,z), and the position information of the virtual user in the virtual space in the display state of Figure 4(b) is set to (x,y+1,z), thus setting it to a fixed point.

[0063] In other words, when performing fixed-point movement, even if the distance traveled by a finger or other object during a swipe operation in the display state shown in Figure 4(a) is short, the stationary position of the virtual user in the virtual space is not set to, for example, (x+0.3, y, z). When performing fixed-point movement, a variety of movements are not planned, which has the advantage of simplifying transition control and reducing the load on the communication system.

[0064] The settings for performing fixed-point movement are suitable when the object of viewing is a two-dimensional image, such as painting image 600B. This is evident from the fact that when viewing a two-dimensional painting in real space, one usually views it while facing the artwork directly.

[0065] On the other hand, when the object of appreciation is an image of a three-dimensional work, such as a sculpture, when appreciating a three-dimensional sculpture in real space, one usually does not only view the work while facing it directly, but also from various angles. Therefore, a setting for fixed-point movement is not suitable.

[0066] If no settings are configured for fixed-point movement, for example, in the display state shown in Figure 4(a), if the user performs a swipe operation, moving their finger or other object touching the touch panel display of the client device 200 to the left, for a relatively short period of time, the virtual user's stationary position may be, for example, (x+0.5, y, z). If the operation is performed for a relatively long period of time, the virtual user's stationary position may be, for example, (x+1.5, y, z). Of course, it is also possible to control whether or not to uniformly set the position information of the virtual user's stationary position in the virtual space, regardless of whether the object of viewing is an image of a two-dimensional work or an image of a three-dimensional work.

[0067] Furthermore, even when viewing images of three-dimensional artwork, if the system is configured to allow fixed-point movement, it is also possible to enable rotation of the viewing object. This has the advantage of reducing the load on the communication system while allowing users to view the artwork's image from various angles.

[0068] Figure 6 shows an example of an image that will pop up after transitioning to the display state shown in Figure 5(b) or Figure 5(c). Of course, instead of displaying the image example as a pop-up, the display area for displaying it may be superimposed on the background image, so that the image example shown in Figure 6 is displayed in response to the user's actions.

[0069] Figure 6 shows the artwork image display area 710, such as the painting image 600C in Figure 5(c), the avatar image display area 720 of the virtual museum's commentator, the text display area 730 of the commentator's explanation about the artist and artwork related to the painting image 600C, and the avatar image display area 740 of other users, which is displayed in the state shown in Figure 5(b) or Figure 5(c). The avatar image display area 720 and the avatar image display area 740 are associated with a unique ID assigned to the commentator and a unique ID assigned to the seven other users, respectively.

[0070] If the pop-up image shown in Figure 6 is displayed, and the microphone 430 is turned on, the user can communicate online with the presenter or other users.

[0071] For example, if a user operates the first client device 200A, a commentator operates the second client device 200B, and in this example, seven other users each operate the third client device 200C to the ninth client device 200I, the first client device 200A will pick up the user's voice using the microphone 430, include the IDs assigned to the commentator and the seven other users (as shown in Figure 6) as destinations in the voice signal, and then transmit it to the server device 100 via the network 300.

[0072] When the server device 100 receives an audio signal transmitted from the first client device 200A, it generates a control signal related to that audio signal and transmits it to the second client device 200B through the ninth client device 200I via the network 300.

[0073] When the second client device 200B to the ninth client device 200I receive the control signal, they generate an audio signal and output it to the speaker 520. Therefore, the audio signal based on the user's voice is not transmitted to the client device 200 of a user in another virtual booth who is viewing a painting image 600B, and vice versa.

[0074] Furthermore, for example, a user operating the first client device 200A may want to have a private online conversation with a commentator. Therefore, when the user taps the avatar image display area 720, only the ID assigned to the commentator will be switched to the above destination.

[0075] Similarly, when a user taps on the avatar image of one or more other users they wish to converse with among the avatar images displayed in the avatar image display area 740, the destination will switch to only the ID assigned to those other users.

[0076] Furthermore, as the number of other users displayed in the avatar image display area 740 increases, the number of client devices 200 connected to the server device 100 also increases. Therefore, in order to reduce the communication load, the amount of control signals sent from the server device 100 to each client device 200 should be reduced. To achieve this, the amount of control signals sent to each client device 200 should be reduced uniformly, for example.

[0077] This reduces the amount of control signals transmitted from the server device 100 to the client device 200, thereby suppressing an increase in communication traffic on the network 300 connecting them, and reducing the time lag between when an audio signal is transmitted from one client device 200 to the server device 100 and when a control signal based on that audio signal is received by another client device 200.

[0078] In this embodiment, the communication system was primarily described using the example of its application to a virtual art museum. However, it can be suitably applied to virtual spaces such as virtual museums, virtual zoos, virtual offices, virtual concert venues, virtual live venues, and virtual cinemas that exist on the internet. [Brief explanation of the drawing]

[0079] [Figure 1] This is a block diagram showing a schematic configuration of a communication system in a virtual space according to an embodiment of the present invention. [Figure 2] This is a block diagram showing the schematic configuration of the server device 100 shown in Figure 1. [Figure 3] This block diagram shows a schematic configuration of the client device 200 shown in Figure 1. [Figure 4] This figure shows an example of an image displayed on the display 510 of the first client device 200A shown in Figure 1. [Figure 5] This figure shows an example of an image displayed on the display 510 of the first client device 200A shown in Figure 1. [Figure 6] This figure shows an example of an image displayed on the display 510 of the first client device 200A shown in Figure 1. [Explanation of Symbols]

[0080] 100 Server Devices 110 Video signal control means 120 Audio signal control means 130 Transmission amount control means 140 Content storage means 150 Calculation method 160 databases 200 client devices 210 Video signal processing means 220 Audio signal processing means 230 Reception methods 240 Transmission methods 300 Networks 400 Input Devices 410 Keyboard 420 mice 430 Microphones 500 Output Device 510 displays 520 speakers 600A~600C Painting images 600D Switchable Display Area 710 Work image display area 720 Avatar image display area 730 Text display area 740 avatar image display area

Claims

1. A virtual space system comprising a handheld client device having a touch panel display operated by a user, The aforementioned client device The means for displaying a portion of an image in a virtual space on the aforementioned touch panel display, A means for receiving a swipe operation performed by the user, Means for transitioning the display area of ​​the image according to the swipe direction of the swipe operation, It has, A storage means for linking and storing virtual booths within the virtual space with the contact information of the person in charge of virtual user support in said virtual booth, A detection means for detecting when a virtual user in the virtual space has entered the virtual booth, A notification means that, when the arrival of the virtual booth is detected by the detection means, notifies the contact person corresponding to the virtual booth stored by the storage means, A virtual space system equipped with these features.

2. The virtual space system according to claim 1, wherein the transition of the display area is performed in a manner in which a virtual user moves along a virtual pathway.

3. The virtual space system according to claim 1, wherein the virtual space is displayed using a web browser.

4. A server device connected to the aforementioned client device via a network, Equipped with, The virtual space system according to claim 1.

Images