Message sending device and message receiving device

Abstract

Provided is a terminal device comprising: a trajectory information generation unit that generates trajectory information about a trajectory of a virtual object corresponding to a message when the virtual object is moved in a virtual space in response to an operation by a user of a transmission source; a transmission information generation unit that generates transmission information including message information indicating the message and the trajectory information; and a transmission control unit that uses a communication device to transmit the transmission information to a transmission destination designated by the user.

Description

[Technical Field]

[0001] The present invention relates to a message sending device that sends a message and a message receiving device that receives a message. [Background technology]

[0002] Non-Patent Document 1 discloses that when sending and receiving emails, an animation of a character delivering email is displayed. With this technology, when a user sends email, an animation of an animal character carrying the email and leaving the room is displayed on the display. When a user receives email, an animation of a character visiting with the email is displayed on the display.

[0003] In XR technology, which includes VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality), a three-dimensional virtual space is displayed on XR glasses worn by a user. A virtual object associated with a message may be displayed in this virtual space. When the technology described in Non-Patent Document 1 is applied to a three-dimensional virtual space, an animation of a character delivering a virtual object is displayed on the XR glasses. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] "Let's play with PostPet, the popular email software!", [online], October 13, 1997, Watch Editorial Department INTERNET, [Retrieved January 24, 2022], Internet<URL:https: / / internet.watch.impress.co.jp / www / article / 971013 / special.htm> Summary of the Invention [Problem to be solved by the invention]

[0005] However, the action of the character delivering the virtual object is a predetermined action, and therefore the sending user cannot control the movement of the virtual object in the virtual space that is viewed by the receiving user.

[0006] The present disclosure aims to provide a message sending device and a message receiving device that allow a user who sends a message to control the movement of a virtual object in a virtual space that is viewed by a user who is a destination of the message. [Means for solving the problem]

[0007] A preferred aspect of the present invention provides a message sending device including: a trajectory information generation unit that generates trajectory information regarding the trajectory of a virtual object corresponding to a message when the virtual object is moved in a virtual space in response to an operation by a user who is the sender of the message; a transmission information generation unit that generates transmission information including message information indicating the message and the trajectory information; and a transmission control unit that uses a communication device to send the transmission information to a destination specified by the user.

[0008] A preferred aspect of the present invention is a message receiving device that includes a receiving control unit that causes a communication device to receive the transmission information transmitted from a message transmitting device, an image generating unit that generates an image of a virtual object moving in a virtual space in accordance with the trajectory information included in the transmission information, and a display control unit that displays the generated image on a display device of the destination user. [Effects of the Invention]

[0009] According to the present invention, a user who has sent a message can control the movement of a virtual object in a virtual space that is visually recognized by a user who has sent the message. [Brief explanation of the drawings]

[0010] [Figure 1]1 is a diagram showing the overall configuration of an information processing system 1 according to an embodiment. [Figure 2] FIG. 10 is a schematic diagram showing an example of a virtual space VS visually recognized by a sending user U[K] through XR glasses 20-K. [Figure 3] FIG. 2 is a perspective view showing the appearance of XR glasses 20-K according to an embodiment. [Figure 4] FIG. 2 is a block diagram showing an example of the configuration of XR glasses 20-K according to an embodiment. [Figure 5] FIG. 2 is a block diagram showing an example of the configuration of a terminal device 10-K according to the embodiment. [Figure 6] FIG. 2 is an explanatory diagram showing an example of the information structure of reference trajectory information D. [Figure 7] FIG. 3 is a schematic diagram showing the first to seventh reference trajectories Sr1 to Sr7. [Figure 8] FIG. 3 is a functional block diagram of a trajectory information generating unit 113. [Figure 9] FIG. 10 is an explanatory diagram showing an example of a user trajectory Su1 before correction and a user trajectory Su2 after correction. [Figure 10A] FIG. 2 is a schematic diagram showing an example of an orbit Sx obtained by combining the orbit S and the first reference orbit Sr1. [Figure 10B] FIG. 3 is a schematic diagram showing an example of an orbit Sy obtained by combining the first reference orbit Sr1 and the third reference orbit Sr3. [Figure 11] FIG. 10 is a schematic diagram showing an example of a virtual space VS in a case where a user U[K] selects one or more trajectories from among a plurality of trajectories. [Figure 12] FIG. 2 is a block diagram showing an example of the configuration of a server 30. [Figure 13] 10 is a flowchart showing an example of the operation of a transmission process of a terminal device 10-K according to the embodiment. [Figure 14A] FIG. 10 is a schematic diagram showing an example of association between message information and a virtual object VO. [Figure 14B] FIG. 2 is a schematic diagram showing an example of a virtual space VS related to generation of a trajectory of a virtual object VO. [Figure 15] 10 is a flowchart showing an example of processing content related to generation of trajectory information. [Figure 16A]FIG. 10 is a schematic diagram showing an example of a virtual space VS for receiving a user trajectory input. [Figure 16B] FIG. 10 is a schematic diagram showing an example of a virtual space VS for receiving instructions regarding trajectory synthesis. [Figure 17] 10 is a flowchart showing an example of the operation of a transmission process of a terminal device 10-K according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] 1: Embodiment The information processing system 1 will be described below with reference to FIGS.

[0012] 1.1: Configuration of the embodiment 1.1.1: Overall structure FIG. 1 is a block diagram showing the overall configuration of an information processing system 1. As shown in FIG. 1, the information processing system 1 includes terminal devices 10-1, 10-2, ... 10-K, ... 10-J, XR glasses 20-1, 20-2, ... 20-K, ... 20-J, and a server 30. J is an integer equal to or greater than 1. K is an integer equal to or greater than 1 and equal to or less than J. In this embodiment, the terminal devices 10-1, 10-2, ... 10-K, ... 10-J have the same configuration. However, terminal devices that do not have the same configuration as the other terminal devices may be included. In this embodiment, the XR glasses 20-1, 20-2, ... 20-K, ... 20-J have the same configuration. However, XR glasses that do not have the same configuration as the other XR glasses may be included.

[0013] In the information processing system 1, the terminal device 10-K and the server 30 are communicably connected to each other via a communication network NET. The terminal device 10-K and the XR glasses 20-K are also communicably connected to each other. In FIG. 1, the user U[K] uses a combination of the terminal device 10-K and the XR glasses 20-K. Users U[1], U[2], ... U[K-1], U[K+1], ... U[J] also use combinations of a terminal device and XR glasses. The terminal device 10-K is an example of a message sending device and a message receiving device.

[0014] The server 30 provides various data and cloud services to the terminal device 10-K via the communication network NET. The cloud services include services related to sending and receiving messages.

[0015] The terminal device 10-K displays virtual objects arranged in a virtual space on the XR glasses 20-K worn on the head of the user U[K]. The virtual space is a three-dimensional space. The virtual objects are expressed in three dimensions. Examples of the virtual objects include virtual objects representing data such as still images, videos, 3DCG models, HTML files, and text files, as well as virtual objects representing applications. Here, examples of text files include memos, source code, diaries, and recipes. Examples of applications include browsers, applications for using SNS, and applications for generating document files. The terminal device 10-K is preferably a mobile terminal device such as a smartphone or a tablet.

[0016] The XR glasses 20-K are a see-through wearable display worn on the head of the user U[K]. The XR glasses 20-K display virtual objects on display panels provided in each of the lenses for both eyes based on the control of the terminal device 10-K. The XR glasses 20-K are an example of a display device.

[0017] In this embodiment, a user U[K] wearing XR glasses 20-K on his / her head uses the terminal device 10-K to send message information indicating a message to, for example, the terminal device 10-1 used by another user U[1]. The message may be, for example, text, audio, or a moving image such as an animation.

[0018] FIG. 2 shows an example of a virtual space VS viewed by a sender user U[K] through the XR glasses 20-K. In FIG. 2, the X-axis, Y-axis, and Z-axis are perpendicular to one another. In the following description, the positive direction along the X-axis is referred to as the X-positive direction, the negative direction along the X-axis is referred to as the X-negative direction, the positive direction along the Y-axis is referred to as the Y-positive direction, the negative direction along the Y-axis is referred to as the Y-negative direction, the positive direction along the Z-axis is referred to as the Z-positive direction, and the negative direction along the Z-axis is referred to as the Z-negative direction. In the virtual space VS shown in FIG. 2, a virtual object VO corresponding to a message from user U[K] exists. User U[K] can send message information by specifying a destination. In this example, the virtual object VO has a spherical shape. Furthermore, the virtual object VO moves within the virtual space VS in response to an operation by user U[K]. The virtual object VO shown in FIG. 2 moves along a trajectory S. For example, user U[K] moves the virtual object VO by holding it in one hand and moving it around his or her body. In FIG. 2, the user U[K] is omitted.

[0019] In this embodiment, trajectory information regarding a trajectory S along which a source user moves a virtual object VO in a virtual space VS is associated with message information and transmitted to a destination user U[1]. When the user U[1] views the virtual space VS using the XR glasses 20-1, the virtual object VO moves in the virtual space VS based on the trajectory information. According to the information processing system 1, the source user U[K] can control the behavior of the virtual object VO in the virtual space viewed by the destination user U[1]. The user U[1] is an example of a destination user.

[0020] 1.1.2: XR Glasses Configuration Fig. 3 is a perspective view showing the appearance of the XR glasses 20-K. As shown in Fig. 2, the appearance of the XR glasses 20-K is similar to that of ordinary eyeglasses, and includes temples 91 and 92, a bridge 93, frames 94 and 95, and lenses 41L and 41R.

[0021] An imaging device 26 is provided on the bridge 93. The imaging device 26 captures images of the outside world. The imaging device 26 also outputs imaging information indicating the captured image. A left depth detection device 29L is provided on the frame 94. A right depth detection device 29R is provided on the frame 95. The depth detection devices 29L and 29R output depth information indicating the distance to an object existing in real space.

[0022] Each of the lenses 41L and 41R is provided with a half mirror. The frame 94 is provided with a liquid crystal panel or organic EL panel for the left eye and an optical member that guides light emitted from the display panel for the left eye to the lens 41L. The liquid crystal panel or organic EL panel will hereinafter be collectively referred to as the display panel. The half mirror provided in the lens 41L transmits external light and guides it to the left eye, and reflects the light guided by the optical member, and then causes the reflected light to enter the left eye. The frame 95 is provided with a display panel for the right eye and an optical member that guides light emitted from the display panel for the right eye to the lens 41R. The half mirror provided in the lens 41R transmits external light and guides it to the right eye, and also reflects the light guided by the optical member, and then causes the reflected light to enter the right eye.

[0023] The display 28, which will be described later, includes a lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as a lens 41R, a display panel for the right eye, and an optical member for the right eye.

[0024] In the above configuration, the user U[K] can observe the image displayed on the display panel in a see-through state, superimposed on the outside world. Furthermore, the XR glasses 20-K display the left-eye image on the left-eye display panel and the right-eye image on the right-eye display panel, among the binocular images with parallax. Therefore, the XR glasses 20-K allow the user U[K] to perceive the displayed image as if it had depth and a three-dimensional effect.

[0025] FIG. 4 is a block diagram showing an example configuration of the XR glasses 20-K. The XR glasses 20-K include a processing device 21, a storage device 22, a gaze detection device 23, a GPS device 24, a motion detection device 25, an imaging device 26, a communication device 27, a display 28, and depth detection devices 29L and 29R. The elements of the XR glasses 20-K are connected to each other by one or more buses for communicating information. Note that the term "device" in this specification may be replaced with other terms such as circuit, device, or unit.

[0026] The processing device 21 is a processor that controls the entire XR glasses 20-K. The processing device 21 is configured using, for example, one or more chips. The processing device 21 is also configured using, for example, a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 21 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 21 executes various processes in parallel or sequentially.

[0027] The storage device 22 is a recording medium that can be read from and written to by the processing device 21. The storage device 22 also stores a plurality of programs including a control program PR1 that the processing device 21 executes.

[0028] After detecting the gaze of the user U[K], the gaze detection device 23 generates gaze information indicating the detection result. Any method may be used for the gaze detection device 23 to detect the gaze. The gaze detection device 23 may generate the gaze information based on the position of the inner corner of the eye and the position of the iris, for example. The gaze information indicates the direction of the gaze of the user U[K]. The gaze detection device 23 outputs the gaze information to the processing device 21. The gaze information output to the processing device 21 is output to the terminal device 10-K via the communication device 27.

[0029] The GPS device 24 receives radio waves from multiple satellites. The GPS device 24 also generates user position information indicating the position of the user U[K] from the received radio waves. The user position information may be in any format as long as it can identify the position of the user U[K]. The user position information indicates, for example, the latitude and longitude of the XR glasses 20-K. The user position information is output to the processing device 21. The processing device 21 outputs the user position information to the terminal device 10-K via the communication device 27.

[0030] The motion detection device 25 detects the movement of the XR glasses 20-K. The motion detection device 25 includes inertial sensors, such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration. The acceleration sensor detects acceleration along each of the orthogonal X, Y, and Z axes. The gyro sensor detects angular acceleration around each of the X, Y, and Z axes as the central axis of rotation. The motion detection device 25 can generate user posture information indicating the posture of the XR glasses 20-K based on the output information of the gyro sensor. The user movement information includes acceleration information indicating the acceleration along each of the three axes and angular acceleration information indicating the angular acceleration along each of the three axes. The motion detection device 25 also outputs the user posture information indicating the posture of the XR glasses 20-K and user movement information related to the movement of the XR glasses 20-K to the processing device 21. The user posture information and user movement information output to the processing device 21 are then output to the terminal device 10-K via the communication device 27.

[0031] The imaging device 26 outputs imaging information obtained by capturing an image of the outside world. The imaging device 26 also includes, for example, a lens, an imaging element, an amplifier, and an AD converter. Light collected through the lens is converted into an imaging signal, which is an analog signal, by the imaging element. The amplifier amplifies the imaging signal and outputs the amplified imaging signal to the AD converter. The AD converter converts the amplified imaging signal, which is an analog signal, into imaging information, which is a digital signal. The converted imaging information is output to the processing device 21. The imaging information output to the processing device 21 is output to the terminal device 10-K via the communication device 27. The imaging device 26 is, for example, a camera.

[0032] The communication device 27 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 27 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 27 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 27 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0033] The display 28 is a device that displays images. The display 28 displays various images under the control of the processing device 21. As described above, the display 28 includes the lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as the lens 41R, a display panel for the right eye, and an optical member for the right eye. As the display panel, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used.

[0034] 1.1.3: Terminal device configuration 5 is a block diagram showing an example of the configuration of the terminal device 10-K. The terminal device 10-K includes a processing device 11, a storage device 12, a communication device 13, a display 14, an input device 15, and an inertial sensor 16. The elements of the terminal device 10-K are connected to each other by one or more buses for communicating information.

[0035] The processing device 11 is a processor that controls the entire terminal device 10-K. The processing device 11 is configured, for example, using one or more chips. The processing device 11 is configured, for example, using a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 11 may be realized by hardware such as a DSP, ASIC, PLD, or FPGA. The processing device 11 executes various processes in parallel or sequentially.

[0036] The storage device 12 is a recording medium that can be read from and written to by the processing device 11. The storage device 12 also stores a plurality of programs including a control program PR2 executed by the processing device 11. The storage device 12 may further store external image information indicating an image to be displayed on the XR glasses 20-K. This external image information includes external image information indicating a virtual object VO associated with a message.

[0037] Furthermore, the storage device 12 stores reference trajectory information D. The reference trajectory information D indicates a plurality of predetermined trajectories of the virtual object VO. As described with reference to FIG. 2, the user U[K] can move the virtual object VO in the virtual space VS by directly manipulating the virtual object VO. The user U[K] can also select one trajectory from a plurality of predetermined trajectories. In this case, the virtual object VO moves along the selected trajectory. As shown in FIG. 6, the reference trajectory information D in this example includes first reference trajectory information D1 indicating a first reference trajectory Sr1, second reference trajectory information D2 indicating a second reference trajectory Sr2, third reference trajectory information D3 indicating a third reference trajectory Sr3, fourth reference trajectory information D4 indicating a fourth reference trajectory Sr4, fifth reference trajectory information D5 indicating a fifth reference trajectory Sr5, sixth reference trajectory information D6 indicating a sixth reference trajectory Sr6, and seventh reference trajectory information D7 indicating a seventh reference trajectory Sr7.

[0038] FIG. 7 is a schematic diagram showing the first to seventh reference trajectories Sr1 to Sr7. The first reference trajectory Sr1 is a trajectory that moves the virtual object VO in the positive Z direction, i.e., upward in the drawing. The second reference trajectory Sr2 is a trajectory that moves the virtual object VO in the negative Z direction, i.e., downward in the drawing. The third reference trajectory Sr3 is a trajectory that moves the virtual object VO in the positive X direction, i.e., rightward in the drawing. The fourth reference trajectory Sr4 is a trajectory that moves the virtual object VO in the negative X direction, i.e., leftward in the drawing. The fifth reference trajectory Sr5 is a trajectory that moves the virtual object VO in a wavy manner in the positive X direction. The sixth reference trajectory Sr6 is a trajectory that moves the virtual object VO in a wavy manner in the negative X direction. The seventh reference trajectory Sr7 is a trajectory that moves the virtual object VO in a floating manner in the virtual space VS.

[0039] The communication device 13 shown in Fig. 5 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 13 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 13 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 13 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0040] The display 14 is a device that displays images and text information. The display 14 displays various images under the control of the processing device 11. For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are suitably used as the display 14.

[0041] The input device 15 generates operation information according to an operation performed by the user U[K] on the input device 15. The input device 15 outputs the operation information to the processing device 11. For example, the input device 15 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 15 includes a touch panel, it may also serve as the display 14.

[0042] The inertial sensor 16 is a sensor that detects inertial force. The inertial sensor 16 includes, for example, one or more sensors selected from an acceleration sensor, an angular velocity sensor, and a gyro sensor. The inertial sensor 16 outputs sensor information indicating the detection results of the one or more sensors to the processing device 11. The processing device 11 detects the attitude of the terminal device 10-K based on the sensor information. Furthermore, the processing device 11 accepts the selection of a virtual object VO, the input of characters, and the input of instructions in the virtual space VS based on the attitude of the terminal device 10-K. For example, when the user U[K] operates the input device 15 with the central axis of the terminal device 10-K facing a predetermined area in the virtual space VS, the virtual object VO located in the predetermined area is selected. The operation of the user U[K] on the input device 15 is, for example, a double tap. In this way, the user U[K] can select a virtual object VO without looking at the input device 15 of the terminal device 10-K by operating the terminal device 10-K. Furthermore, when inputting characters, the processing device 11 displays a virtual keyboard having multiple virtual keys in the virtual space. The user U[K] inputs characters by operating the input device 15 while pointing the central axis of the terminal device 10-K toward the virtual key. By this input operation, the user U[K] can input the address of the destination of the message.

[0043] The processing device 11 reads the control program PR2 from the storage device 12 and executes the read control program PR2. As a result, the processing device 11 functions as an acquisition unit 111, a reception unit 112, a trajectory information generation unit 113, a transmission information generation unit 114, a display control unit 115, a transmission control unit 116, and a reception control unit 117.

[0044] The acquisition unit 111 acquires information including imaging information, line-of-sight information, user position information, user posture information, user movement information, and depth information via the communication device 13.

[0045] The reception unit 112 receives operation information output from the input device 15 and sensor information output from the inertial sensor 16. The reception unit 112 generates device attitude information indicating the attitude of the terminal device 10-K based on the sensor information.

[0046] When a virtual object VO corresponding to a message is moved in a virtual space in response to an operation by the user U[K], the trajectory information generation unit 113 generates trajectory information relating to the trajectory of the virtual object VO. It is assumed that the user U[K] is the user who sent the message. The operation by the user U[K] includes, in addition to an operation on the input device 15, an operation based on the line of sight of the user U[K] and a gesture by the user U[K]. An operation based on the line of sight is identified by line of sight information acquired from the XR glasses 20-K. An operation based on a gesture is identified by imaging information acquired from the XR glasses 20-K.

[0047] 8 is a block diagram showing a detailed configuration of the trajectory information generating unit 113. The trajectory information generating unit 113 includes a user trajectory generating unit 113A, a reference trajectory selecting unit 113B, and a trajectory synthesizing unit 113C.

[0048] The user trajectory generation unit 113A generates, as a user trajectory, a trajectory of the virtual object VO that moves when the user U[K] directly operates the virtual object VO corresponding to the message. The user U[K]'s direct operation on the virtual object VO is an example of a second operation. The user U[K]'s operation on the virtual object VO is, for example, an operation in which the user U[K] holds the virtual object VO in his / her hand and moves the virtual object VO in the virtual space VS. When this operation is performed, the user trajectory generation unit 113A identifies the trajectory of the virtual object VO by tracking the position of the user U[K]'s hand. The user trajectory generation unit 113A identifies the position of the user U[K]'s hand based on depth information. Alternatively, the user trajectory generation unit 113A identifies the position of the user U[K]'s hand based on imaging information. In identifying the position of the hand based on imaging information, the user trajectory generation unit 113A may identify the position of the hand using a learning model that has learned the relationship between the captured image and the hand position.

[0049] The user trajectory generation unit 113A generates trajectory information indicating a change over time in the position of the hand of the user U[K]. Relative position information indicating the relative positional relationship between the position of the hand of the user U[K] and the center position of the virtual object VO is known. Based on the trajectory information and the relative position information, the user trajectory generation unit 113A calculates a user trajectory indicating a change over time in the position of the virtual object VO in the virtual space VS, and generates user trajectory information indicating the calculated user trajectory. The trajectory S shown in FIG. 2 is an example of a user trajectory.

[0050] Incidentally, when the user U[K] directly operates the virtual object VO, the user U[K]'s hand may shake. In this case, the user trajectory may be slightly shaken. Therefore, the user trajectory generation unit 113A may correct the user trajectory by performing a process to remove noise components associated with the operation of the user U[K] from the user trajectory. When the correction is performed, the user trajectory information generated by the user trajectory generation unit 113A indicates the corrected user trajectory. For example, the user trajectory Su1 shown in FIG. 9 is the trajectory before correction. The user trajectory generation unit 113A generates a corrected user trajectory Su2 by performing a process to remove noise components from the user trajectory Su1. The process to remove noise components is, for example, a low-pass filter process that removes high-frequency components from the user trajectory. However, the process to remove noise components is not limited to a low-pass filter process, and any process that can remove noise may be used.

[0051] The user U[K] may specify a user trajectory as the trajectory of the virtual object VO by directly manipulating the virtual object VO, or may specify the trajectory of the virtual object VO by selecting one trajectory from multiple trajectories prepared in advance. In this embodiment, the multiple trajectories prepared in advance are a first reference trajectory Sr1, a second reference trajectory Sr2, ..., and a seventh reference trajectory Sr7 shown in FIG. 7. Furthermore, the trajectory information generation unit 113 may generate a synthesized trajectory by synthesizing the user trajectory and one or more selected trajectories, and generate trajectory information indicating the synthesized trajectory. The synthesis of the trajectories may be a synthesis of two or more reference trajectories, or may be a synthesis of the user trajectory and one or more reference trajectories. The operation of selecting one or more trajectories from multiple trajectories prepared in advance is an example of a first operation.

[0052] When the operation of the user U[K] is a first operation, the reference trajectory selection unit 113B identifies one or more trajectories selected by the first operation. Specifically, the reference trajectory selection unit 113B reads out, from the storage device 12, reference trajectory information D corresponding to the one or more trajectories selected by the first operation.

[0053] The trajectory synthesis unit 113C synthesizes two or more trajectories to generate a synthesized trajectory. When the user U[K] performs a second operation to move the virtual object VO in the virtual space VS, a user trajectory is generated. When one or more trajectories are selected by the reference trajectory selection unit 113B while the user trajectory is being generated, the trajectory synthesis unit 113C synthesizes the user trajectory with the selected one or more trajectories. For example, when the user trajectory is the loop-shaped trajectory S shown in FIG. 2 and the selected trajectory is the first reference trajectory Sr1 shown in FIG. 7, the trajectory synthesis unit 113C synthesizes the trajectory S with the first reference trajectory Sr1 to generate the trajectory Sx shown in FIG. 10A.

[0054] Furthermore, when one or more trajectories are selected by the reference trajectory selection unit 113B in a state where a user trajectory has not been generated, the trajectory synthesis unit 113C synthesizes one or more trajectories. For example, when the reference trajectory selection unit 113B selects the first reference trajectory Sr1 shown in Fig. 7 and then selects the third reference trajectory Sr3, the trajectory synthesis unit 113C synthesizes the first reference trajectory Sr1 and the third reference trajectory Sr3 to generate the trajectory Sy shown in Fig. 10B.

[0055] 5 generates transmission information including the message information, the trajectory information generated by the trajectory information generation unit 113, the destination information, and the source information. The destination information is, for example, an address indicating the destination of the message. The source information is, for example, an address indicating the source of the message.

[0056] The display control unit 115 generates internal image information indicating an image to be displayed on the display 14 and external image information indicating an image to be displayed in the XR glasses 20-K. The display control unit 115 outputs the internal image information to the display 14. The display control unit 115 outputs the external image information to the XR glasses 20-K via the communication device 13, thereby causing the XR glasses 20-K to display a virtual space VS including a virtual object VO.

[0057] For example, when the user U[K] selects one or more trajectories from a plurality of trajectories, the display control unit 115 generates an image of the virtual space VS shown in Fig. 11. As shown in Fig. 11, a first icon A1, a second icon A2, ..., a seventh icon A7 are arranged in the virtual space VS in association with the virtual object VO. The first icon A1, the second icon A2, ..., the seventh icon A7 correspond one-to-one to the first reference trajectory Sr1, the second reference trajectory Sr2, ..., the seventh reference trajectory Sr7. The user U[K] can specify one or more reference trajectories by selecting an icon.

[0058] 5 causes the communication device 13 to transmit the transmission information to a destination specified by the user U[K]. Specifically, the transmission information including the destination information is transmitted to the server 30.

[0059] The reception control unit 117 causes the communication device 13 to receive the transmission information transmitted from the terminal device of the source user via the server 30. When the user U[1] is the source user and the user U[K] is the destination user, the transmission information transmitted from the terminal device 10-1 is received by the terminal device 10-K via the server 30.

[0060] The image generation unit 118 generates a virtual object VO corresponding to the message. The image generation unit 118 also generates an image in which the virtual object VO moves in the virtual space VS according to the trajectory information included in the received transmission information. The image generation unit 118 outputs external image information indicating the generated image to the display control unit 115. The display control unit 115 outputs the external image information to the XR glasses 20-K of the user U[K], who is the destination user. If the user U[1] is the source user, the behavior of the virtual object VO moving in the virtual space VS in response to the operation of the user U[1] is displayed in the XR glasses 20-K of the destination user U[K]. This display allows the user U[K] to visually recognize the virtual object VO moving in the virtual space VS through the XR glasses 20-K.

[0061] 1.1.4: Server configuration 12 is a block diagram showing an example of the configuration of the server 30. The server 30 includes a processing device 31, a storage device 32, a communication device 33, a display 34, and an input device 35. The elements of the server 30 are connected to each other by one or more buses for communicating information.

[0062] The processing device 31 is a processor that controls the entire server 30. The processing device 31 is configured, for example, using one or more chips. The processing device 31 is configured, for example, using a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 31 may be realized by hardware such as a DSP, ASIC, PLD, or FPGA. The processing device 31 executes various processes in parallel or sequentially.

[0063] The storage device 32 is a recording medium that can be read and written by the processing device 31. The storage device 32 also stores a plurality of programs including the control program PR3 executed by the processing device 31. The storage device 32 also stores a database DB. The database DB stores identification information, transmission information, reception date and time information, and transmission date and time information in association with one another. The identification information is information that uniquely identifies the transmission information. The transmission information is received from the source terminal devices 10-1 to 10-J. The reception date and time information indicates the date and time when the transmission information was received by the server 30. The transmission date and time information indicates the date and time when the transmission information was transmitted to the destination terminal devices 10-1 to 10-J.

[0064] The communication device 33 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 33 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 33 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 33 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0065] The display 34 is a device that displays images and text information. The display 34 displays various images under the control of the processing device 31. For example, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used as the display 34.

[0066] The input device 35 is a device that receives operations from an administrator of the information processing system 1. For example, the input device 35 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 35 includes a touch panel, it may also serve as the display 34.

[0067] The processing device 31, for example, reads the control program PR3 from the storage device 32 and executes the read control program PR3. As a result, the processing device 31 functions as an acquisition unit 311 and an output unit 312.

[0068] The acquisition unit 311 acquires various types of information including transmission information from the terminal device 10-K via the communication device 33.

[0069] The output unit 312 references the destination information included in the transmission information, and outputs the transmission information via the communication device 33 to the destination address indicated by the destination information.

[0070] 1.2: Operation of the embodiment The operation of the terminal device 10-K will be explained separately for transmission processing and reception processing.

[0071] 1.2.1: Sending process FIG. 13 is a flowchart showing the operation of the transmission process of the terminal device 10-K according to the embodiment.

[0072] In step S1, the processing device 11 functions as an acquisition unit 111. The processing device 11 acquires message information created by the user U[K].

[0073] In step S2, the processing device 11 functions as the display control unit 115 and the image generation unit 118. The processing device 11 generates a virtual object VO corresponding to the message information created in step S1 and displays the generated virtual object VO on the XR glasses 20-K. The user U[K] may perform, for example, the following operations. First, the user U[K] inputs an instruction to generate a virtual object VO for a message by a gesture, a line of sight, or an operation on the input device 15. The processing device 11 accepts the operation and displays the virtual object VO in the virtual space VS. Second, the user U[K] performs an operation of dragging and dropping a virtual object representing a message such as text onto the virtual object VO for transmission. For example, in FIG. 14A , the message information corresponding to "Hello!" is associated with the virtual object VO by an operation of dragging and dropping the virtual object VOt representing "Hello!" onto the virtual object VO for transmission.

[0074] In step S3, the processing device 11 functions as the display control unit 115 and the reception unit 112. The processing device 11 determines whether an operation to move the virtual object VO has been received in the destination device. The processing device 11, for example, displays the virtual space VS shown in FIG. 14B on the XR glasses 20-K. In this example, buttons B1 and B2 are displayed in association with the virtual object VO, along with a message such as "Do you want to generate a trajectory for the virtual object?" The button B1 is used by the user U[K] to input an instruction to move the virtual object VO in the destination device. The button B2 is used by the user U[K] to input an instruction not to move the virtual object VO in the destination device. When the user U[K] operates the button B1, the processing device 11 determines that an operation to move the virtual object VO has been received in the destination device. On the other hand, when the user U[K] operates the button B2, the processing device 11 determines that an operation not to move the virtual object VO has been received in the destination device.

[0075] If the determination result in step S3 is negative, the processing device 11 advances the process to step S5. On the other hand, if the determination result in step S3 is positive, the processing device 11 generates trajectory information (step S4).

[0076] FIG. 15 is a flowchart showing detailed operations of the terminal device 10-K related to the generation of trajectory information. In step S41, the processing device 11 functions as the display control unit 115 and the reception unit 112. The processing device 11 determines whether or not the input of a user trajectory has been received. For example, the processing device 11 displays the virtual space VS shown in FIG. 16A on the XR glasses 20-K. In this example, buttons B3 and B4 are displayed in association with the virtual object VO along with a message such as "Do you want to move the virtual object?" The button B3 is used by the user U[K] to input an instruction to move the virtual object VO. The button B4 is used by the user U[K] to input an instruction not to move the virtual object VO. When the user U[K] operates the button B3, the processing device 11 determines that the input of the user trajectory has been received. On the other hand, when the user U[K] operates the button B4, the processing device 11 determines that the input of the user trajectory has not been received.

[0077] If the determination result of step S41 is positive, the processing device 11 proceeds to step S42. In step S42, the processing device 11 functions as a user trajectory generation unit 113A. When the user U[K] holds a virtual object VO in his / her hand and moves it through the virtual space VS, the processing device 11 identifies the trajectory of the virtual object VO by tracking the position of the user U[K]'s hand. The processing device 11 generates user trajectory information indicating the identified trajectory.

[0078] In step S43, the processing device 11 functions as the display control unit 115 and the reception unit 112. The processing device 11 determines whether an instruction to combine trajectories has been received. The processing device 11, for example, displays the virtual space VS shown in FIG. 16B on the XR glasses 20-K. In this example, buttons B5 and B6 are displayed in association with the virtual object VO along with a message such as "Do you want to combine trajectories?" The button B5 is used by the user U[K] to input an instruction to combine trajectories. The button B6 is used by the user U[K] to input an instruction not to combine trajectories. When the user U[K] operates the button B5, the processing device 11 determines that an instruction to combine trajectories has been received. On the other hand, when the user U[K] operates the button B6, the processing device 11 determines that an instruction to combine trajectories has not been received.

[0079] If the determination result in step S43 is negative, the processing device 11 advances the process to step S46. If the determination result in step S43 is positive, the processing device 11 advances the process to step S44.

[0080] In step S44, the processing device 11 functions as a reference trajectory selection unit 113B, a display control unit 115, and a reception unit 112. The processing device 11 displays, for example, the virtual space VS shown in FIG. 11B on the XR glasses 20-K. This display prompts the user U[K] to select one or more reference trajectories from among a plurality of reference trajectories. The processing device 11 receives the one or more reference trajectories selected by the user U[K].

[0081] In step S45, the processing device 11 functions as a trajectory synthesis unit 113C. The processing device 11 generates a synthesized trajectory based on the reference trajectory information corresponding to the one or more trajectories received in step S44 and the user trajectory information generated in step S42.

[0082] In step S46, the processing device 11 functions as the trajectory information generating unit 113. The processing device 11 generates trajectory information indicating the trajectory along which the virtual object VO moves in the virtual space VS.

[0083] If it is determined in step S41 that the input of the user trajectory has not been received, the processing device 11 proceeds to step S47. In step S47, the processing device 11 receives one or more reference trajectories selected by the user U[K], similar to step S44.

[0084] In step S48, the processing device 11 determines whether or not two or more reference trajectories have been selected by the user U[K].

[0085] If the determination result in step S48 is positive, the processing device 11 advances the process to step S45. In this case, the processing device 11 synthesizes two or more reference trajectories selected by the user U[K].

[0086] On the other hand, if the determination result of step S48 is negative, the processing device 11 proceeds to step S46. In this case, the processing device 11 generates trajectory information indicating one reference trajectory selected by the user U[K]. The above is the detailed processing content of step S4 shown in FIG. 13.

[0087] 13, the processing device 11 functions as the reception unit 112. The processing device 11 receives a destination designated by the user U[K]. Through this reception, the processing device 11 obtains destination information.

[0088] In step S6, the processing device 11 generates transmission information. Specifically, the processing device 11 generates transmission information including the message information generated in step S1, the trajectory information acquired in step S4, the destination information acquired in step S5, and the source information.

[0089] In step S7, the processing device 11 functions as the transmission control unit 116. The processing device 11 causes the communication device 13 to transmit the transmission information to the destination.

[0090] 1.2.2: Receiving process FIG. 17 is a flowchart showing the operation of the reception process of the terminal device 10-K according to the embodiment.

[0091] In step S11, the processing device 11 functions as the reception control unit 117. The processing device 11 causes the communication device 13 to receive the transmission information, thereby acquiring the transmission information.

[0092] In step S12, the processing device 11 functions as the reception control unit 117. The processing device 11 determines whether or not orbit information is included in the transmission information. If the determination result in step S12 is negative, the processing device 11 ends the reception process.

[0093] If the determination result in step S12 is positive, in step S13, the processing device 11 functions as the image generation unit 118. The processing device 11 generates external image information indicating an image in which the virtual object VO moves in the virtual space VS, based on the trajectory information included in the transmission information.

[0094] In step S14, the processing device 11 functions as the display control unit 115. The processing device 11 outputs the external image information generated in step S13 to the XR glasses 20-K via the communication device 13. This output causes the XR glasses 20-K to display the virtual space VS in which the virtual object VO moves.

[0095] 1.3: Effects of the embodiment According to the above description, the terminal device 10-K as a message sending device includes a trajectory information generation unit 113, a transmission information generation unit 114, and a transmission control unit 116. When a virtual object VO corresponding to a message is moved in the virtual space VS in response to an operation by the sender user U[K], the trajectory information generation unit 113 generates trajectory information regarding the trajectory of the virtual object VO. The transmission information generation unit 114 generates transmission information including message information indicating the message and the trajectory information. The transmission control unit 116 causes the communication device 13 to transmit the transmission information to a destination specified by the user U[K].

[0096] The terminal device 10-K transmits transmission information including message information and trajectory information regarding the trajectory of the virtual object VO to the destination, so that the destination device can reproduce the movement of the virtual object VO that moves in response to the operation of the sender user U[K] based on the trajectory information. Thus, the message sender user U[K] can control the movement of the virtual object VO in the virtual space visually recognized by the message destination user.

[0097] According to the above description, the trajectory indicates a change in the position of the virtual object VO over time in the virtual space VS. The trajectory information generation unit 113 generates the trajectory information by performing a process on the trajectory to remove noise components associated with the operation of the user U[K].

[0098] When a user U[K] touches a virtual object VO in a virtual space VS to move the virtual object VO, the shaking of the user U[K] is reflected in the trajectory of the virtual object VO. The terminal device 10-K executes a process to remove noise components, thereby removing the shaking components of the user U[K] reflected in the trajectory. This allows the movement of the virtual object VO in the virtual space viewed by the destination user to be smooth.

[0099] According to the above description, the operation of the sender user U[K] includes a first operation in which the sender user U[K] selects one or more trajectories from a plurality of trajectories prepared in advance as trajectories of the virtual object VO, and a second operation in which the sender user U[K] moves the virtual object VO in the virtual space VS. The trajectory information generation unit 113 generates, as trajectory information, information indicating a trajectory obtained by combining the one or more trajectories selected by the first operation and the trajectory of the virtual object specified by the second operation.

[0100] The terminal device 10-K has the above configuration, and thus can impart effects to the trajectory of the virtual object specified by the second operation. Furthermore, the source user U[K] can select one or more trajectories from multiple trajectories, thereby generating a variety of trajectories of the virtual object.

[0101] According to the above description, the terminal device 10-K as a message receiving device includes a reception control unit 117 that causes the communication device 13 to receive transmission information transmitted from the message transmitting device, an image generation unit 118 that generates an image of a virtual object VO moving in the virtual space VS in accordance with trajectory information included in the transmission information, and a display control unit 115 that displays the generated image on the XR glasses 20-K of the destination user U[K].

[0102] The terminal device 10-K has the above configuration, and can therefore reproduce the trajectory of a virtual object moved in a virtual space by a source user in the virtual space VS visually recognized by a destination user U[K].

[0103] 3: Variation The present disclosure is not limited to the above-described exemplary embodiments. Specific modified embodiments are exemplified below. Two or more embodiments selected arbitrarily from the following examples may be combined.

[0104] 3.1: Variation 1 In the information processing system 1 according to the embodiment, the terminal device 10-K includes an acquisition unit 111, a reception unit 112, and a trajectory information generation unit 113. However, the server 30 may include the same components as these instead of the terminal device 10-K. Similarly, for the other components, the server 30 may include the same components as these instead of the terminal device 10-K. When the server 30 includes the acquisition unit 111, the reception unit 112, and the trajectory information generation unit 113, various pieces of information acquired by the terminal device 10-K from the XR glasses 20-K are transmitted from the terminal device 10-K to the server 20. Similarly, operation information output from the input device 15 of the terminal device 10-K is also transmitted to the server 30.

[0105] 3.2: Variation 2 In the information processing system 1 according to the embodiment, the terminal device 10-K and the XR glasses 20-K are realized as separate entities. However, the method for realizing the terminal device 10-K and the XR glasses 20-K in the embodiment of the present invention is not limited to this. For example, the XR glasses 20-K may have the same functions as the terminal device 10-K. In other words, the terminal device 10-K and the XR glasses 20-K may be realized in a single housing.

[0106] 3.3: Variation 3 In the terminal device 10-K according to the embodiment, after message information is generated, a virtual object VO associated with the message is generated. However, first, the user U[K] may generate a virtual object VO using the input device 15, and then create message information associated with the virtual object VO.

[0107] 3.4: Variation 4 The information processing system 1 according to the embodiment includes XR glasses 20-K. However, instead of the XR glasses 20-K, the information processing system 1 may include any one of VR glasses employing VR technology, an HMD (Head Mounted Display) employing VR technology, AR glasses employing AR technology, and an HMD employing AR technology. Alternatively, instead of the XR glasses 20-K, the information processing system 1 may include any one of a regular smartphone and a tablet equipped with an imaging device. These VR glasses, AR glasses, HMD, smartphone, and tablet are examples of display devices.

[0108] 4:Other (1) In the above-described embodiment, storage device 12, storage device 22, and storage device 32 are exemplified by ROM and RAM, but may be a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory device (e.g., a card, a stick, a key drive), a CD-ROM (Compact Disc-ROM), a register, a removable disk, a hard disk, a floppy (registered trademark) disk, a magnetic strip, a database, a server, or any other suitable storage medium. The program may also be transmitted from a network via a telecommunications line. The program may also be transmitted from a communications network NET via a telecommunications line.

[0109] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0110] (3) In the above-described embodiment, input and output information may be stored in a specific location (for example, a memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be transmitted to another device.

[0111] (4) In the above-described embodiment, the determination may be made by a value (0 or 1) represented using one bit, by a Boolean value (true or false), or by a comparison of numerical values (e.g., comparison with a predetermined value).

[0112] (5) The order of the process procedures, sequences, flowcharts, etc. illustrated in the above-described embodiments may be rearranged unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0113] (6) Each function illustrated in Figures 1 to 17 is realized by any combination of at least one of hardware and software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized by using one device that is physically or logically coupled, or may be realized by using two or more devices that are physically or logically separated and connected directly or indirectly (for example, by wire, wirelessly, etc.). A functional block may also be realized by combining software with the one device or the multiple devices.

[0114] (7) The programs exemplified in the above-described embodiments should be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, hardware description language, or by other names.

[0115] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0116] (8) In each of the foregoing embodiments, the terms "system" and "network" are used interchangeably.

[0117] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values from a predetermined value, or corresponding other information.

[0118] (10) In the above-described embodiments, the terminal devices 10-1 to 10-J and the server 30 may be mobile stations (MS). A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other appropriate term. In addition, in the present disclosure, terms such as "mobile station," "user terminal," "user equipment (UE)," and "terminal" may be used interchangeably.

[0119] (11) In the above-described embodiments, the terms "connected," "coupled," or any variations thereof refer to any direct or indirect connection or coupling between two or more elements, including the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements are considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0120] (12) In the above embodiments, the phrase "based on" does not mean "based only on," unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0121] (13) As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judged" or "decided." In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." Furthermore, "judgment (decision)" can be interpreted as "assU[M]ing," "expecting," "considering," etc.

[0122] (14) In the above embodiments, when "include," "including," and variations thereof are used, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or" as used in this disclosure is not intended to be an exclusive or.

[0123] (15) In this disclosure, where articles are added by translation, such as a, an, and the in English, this disclosure may include that the nouns following these articles are plural.

[0124] (16) In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

[0125] (17) Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to explicit notification, but may be implicit (e.g., not notifying the predetermined information).

[0126] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure. [Explanation of symbols]

[0127] 1...information processing system, 10-1, 10-2, 10-K, 10-J...terminal device, 11...processing device, 12...storage device, 13...communication device, 14...display, 15...input device, 16...inertial sensor, 20-1, 20-2, 20-K, 20-J...XR glasses, 21...processing device, 23...gaze detection device, 26...imaging device, 27...communication device, 29L, 29R...depth detection device, 30...server, 31...processing device, 32...storage device, 33...communication device, 111...acquisition unit, 112...reception unit, 113...trajectory information generation unit, 114...transmission information generation unit, 115...display control unit, 116...transmission control unit, 117...reception control unit, 118...image generation unit, U[K]...user, VO...virtual object.

Claims

1. a trajectory information generation unit that generates trajectory information regarding a trajectory of a virtual object corresponding to a message when the virtual object is moved in a virtual space in response to an operation by a user who is a sender of the message; a transmission information generating unit that generates transmission information including message information indicating the message and the trajectory information; a transmission control unit that causes a communication device to transmit the transmission information to a destination designated by the user; the operation by the user of the transmission source includes a first operation in which the user of the transmission source selects one or more trajectories from a plurality of trajectories prepared in advance as trajectories of the virtual object, and a second operation in which the user of the transmission source moves the virtual object in the virtual space; the trajectory information generation unit generates, as the trajectory information, information indicating a trajectory obtained by combining the one or more trajectories selected by the first operation and a trajectory of the virtual object designated by the second operation. Message sending device.

2. the trajectory indicates a change in the position of the virtual object in a virtual space over time; The message transmitting device according to claim 1 , wherein the trajectory information generating unit generates the trajectory information by performing a process on the trajectory to remove noise components associated with the user's operation.

3. A reception control unit that causes a communication device to receive the transmission information transmitted from the message transmission device according to claim 1 or 2; an image generation unit that generates an image of a virtual object moving in a virtual space in accordance with the trajectory information included in the transmission information; a display control unit that displays the generated image on a display device of the user of the transmission destination. Message receiving device.

Images