Information Processing Method and Information Processing Apparatus

Abstract

To provide an information processing method and an information processing device that enhance the visibility of display in which a user and another person are superimposed.SOLUTION: An information processing method includes: obtaining a first model that is interlocked with a motion of a first user; obtaining a second model that is interlocked with a motion of a second user; and displaying, while the first user and the second user are continuing a predetermined motion, the first model and the second model at mutually different reference positions.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing method and an information processing apparatus.

Background Art

[0002] A method of superimposing and displaying a three-dimensional image of another person on the body of a user is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a three-dimensional image of another person is superimposed and displayed on the body of a user, the visibility of the user's body or the three-dimensional image of the other person may be reduced.

[0005] In view of such circumstances, an object of the present disclosure is to improve the visibility of a display in which a user and another person are superimposed.

Means for Solving the Problems

[0006] An information processing method according to an embodiment of the present disclosure includes: an information processing apparatus acquiring a first model linked to the operation of a first user; acquiring a second model linked to the operation of a second user; and while the first user or the second user is continuously performing a predetermined operation, displaying the first model and the second model at different reference positions from each other.

[0007] An information processing device according to an embodiment of the present disclosure includes a control unit that acquires a first model linked to a motion of a first user, acquires a second model linked to a motion of a second user, and displays the first model and the second model at different reference positions while the first user or the second user continues a predetermined motion. Effect of the Invention

[0008] According to an information processing method and an information processing device according to an embodiment of the present disclosure, the visibility of a display in which the user and other people are superimposed can be improved. [Brief description of the drawings]

[0009]

Figure 1

Figure 2

Figure 3

[0010] (Example of configuration of information processing system 1) As shown in FIG. 1, an information processing system 1 according to an embodiment includes a first information processing device 10 and a second information processing device 20. When the first information processing device 10 or the second information processing device 20 does not need to be distinguished, it is simply also referred to as an information processing device. It is assumed that the information processing system 1 can provide a service that allows a student to learn by imitating the actions shown by an instructor while watching the actions serving as a model. The learning targets may include, for example, calligraphy, dance, pottery, cooking, or the like. The information processing system 1 according to the present embodiment is assumed to be used to provide a calligraphy lesson. The first information processing device 10 is assumed to be used by a student who learns calligraphy while watching a calligraphy model. The second information processing device 20 is assumed to be used by an instructor who shows the calligraphy model. Among the users of the information processing system 1, the student who uses the first information processing device 10 is also referred to as a first user. The instructor who uses the second information processing device 20 is also referred to as a second user.

[0011] The numbers of the first information processing device 10 and the second information processing device 20 included in the information processing system 1 are not limited to one each, and may be two or more. The first information processing device 10 or the second information processing device 20 may be configured to include a PC such as a notebook PC (Personal Computer), a desktop PC, or a tablet PC. The first information processing device 10 or the second information processing device 20 may also be configured to include a mobile terminal such as a smartphone or a tablet possessed by a user. The first information processing device 10 or the second information processing device 20 is not limited to these examples, and may be configured to include various devices.

[0012] The first information processing device 10 and the second information processing device 20 may be communicably connected to each other via a network, either wired or wirelessly. The first information processing device 10 and the second information processing device 20 may also be communicably connected to each other by wired or wireless communication without going through a network.

[0013] The first information processing apparatus 10 includes a control unit 11, a storage unit 12, and an interface 13. The second information processing apparatus 20 includes a control unit 21, a storage unit 22, and an interface 23. The control unit 11 is also referred to as the first control unit. The control unit 21 is also referred to as the second control unit. The interface 13 is also referred to as the first interface. The interface 23 is also referred to as the second interface.

[0014] The control unit 11 controls at least one component of the first information processing apparatus 10. The control unit 21 controls at least one component of the second information processing apparatus 20. The control units 11 and 21 may be configured identically or similarly. The control unit 11 or 21 may be configured to include one or more processors. In the present embodiment, the "processor" may be a general-purpose processor, a dedicated processor specialized for specific processing, etc., but is not limited thereto. The control unit 11 or 21 may be configured to include one or more dedicated circuits. The dedicated circuit may include, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 11 or 21 may be configured to include a dedicated circuit instead of a processor, or may be configured to include a dedicated circuit together with a processor.

[0015] The storage unit 12 stores any information used for the operation of the first information processing apparatus 10. The storage unit 22 stores any information used for the operation of the second information processing apparatus 20. The storage unit 12 or 22 may be configured identically or similarly. The storage unit 12 or 22 may store a system program, an application program, or the like. The storage unit 12 or 22 may be configured to include, for example, a semiconductor memory, a magnetic memory, an optical memory, or the like, but is not limited thereto and may be configured to include various other memories. The storage unit 12 or 22 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 or 22 may be configured to include an electromagnetic storage medium such as a magnetic disk. The storage unit 12 or 22 may be configured to include a non-transitory computer-readable medium. The storage unit 12 may be included in the control unit 11. The storage unit 22 may be included in the control unit 21.

[0016] The interface 13 outputs information, data, or the like from the control unit 11 or inputs information, data, or the like to the control unit 11 in the first information processing apparatus 10. The interface 23 outputs information, data, or the like from the control unit 21 or inputs information, data, or the like to the control unit 21 in the second information processing apparatus 20. The interface 13 may include a communication module configured to be communicable with the interface 23 of the second information processing apparatus 20 via a network or without passing through a network. The interface 23 may include a communication module configured to be communicable with the interface 13 of the first information processing apparatus 10 via a network or without passing through a network. The communication module may conform to, for example, a mobile communication standard such as 4G (4th Generation) or 5G (5th Generation). The communication module may conform to a communication standard such as LAN (Local Area Network). The communication module may conform to a wired or wireless communication standard. The communication module is not limited thereto and may conform to various communication standards. The interface 13 or 23 may be configured to be connectable to an external communication module.

[0017] The network may be configured to include the Internet, at least one WAN (Wide Area Network), at least one MAN (Metropolitan Area Network), or any combination thereof. The network may also be configured to include at least one wireless network, at least one optical network, or any combination thereof. The wireless network may be configured to include, for example, an ad hoc network, a cellular network, a wireless LAN (Local Area Network), a satellite communication network, or a terrestrial microwave network.

[0018] Interface 13 or 23 may be configured to be compatible with standards such as USB (Universal Serial Bus) or Bluetooth (registered trademark).

[0019] Interface 13 or 23 may be configured to include an input device that receives input such as information or data from a user. The input device may be configured to include, for example, a touch panel or touch sensor, or a pointing device such as a mouse. The input device may be configured to include physical keys. The input device may be configured to include a voice input device such as a microphone. Interface 13 or 23 may also be configured to be connectable to an external input device.

[0020] The interface 13 or 23 may be configured to include an output device that outputs information, data, etc. to the user. The output device may include, for example, a display device that outputs visual information such as images, characters, or graphics. The display device may be configured to include, for example, an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, an inorganic EL display, or a PDP (Plasma Display Panel), etc. The display device is not limited to these displays and may be configured to include various other types of displays. The display device may be configured to include a light-emitting device such as an LED (Light Emitting Diode) or an LD (Laser Diode). The display device may be configured to include various other devices. The output device may include, for example, an audio output device such as a speaker that outputs auditory information such as sound. The output device may include, for example, a vibration device such as a vibrator that outputs tactile information such as vibration. The output device is not limited to these examples and may include various other devices. The interface 13 or 23 may be configured to be connectable to an external output device. The output device may be configured to be wearable by the first user or the second user, such as a wristband, etc. The output device may be configured as a tool used by the first user or the second user during operation practice, such as a brush used in calligraphy, or a kitchen knife or cutting board used in cooking.

[0021] The first information processing device 10 further includes a photographing unit 14, although it is not essential. The second information processing device 20 further includes a photographing unit 24, although it is not essential. The photographing unit 14 is also referred to as the first photographing unit. The photographing unit 24 is also referred to as the second photographing unit. The photographing unit 14 or 24 may include a photographing device such as a camera that photographs an RGB image. The photographing unit 14 or 24 may include a distance measuring device such as a depth sensor or a stereo camera that acquires a depth image. The first information processing device 10 or the second information processing device 20 may acquire an image or distance measuring data (depth data) from an external photographing device or distance measuring device via the interface 13 or 23. The functions of the photographing device or distance measuring device of the photographing unit 14 or 24 may be included in the functions of the interface 13 or 23.

[0022] (Operation example of the information processing system 1) Hereinafter, in the information processing system 1 according to the present embodiment, an operation example in which the first user practices calligraphy while looking at a model by the second user will be described.

[0023] (Acquisition of operation information) The control unit 11 of the first information processing device 10 acquires information regarding the operation of the first user. The information regarding the operation of the first user is also referred to as first operation information. It is assumed that the operation of the first user includes an operation of writing characters in calligraphy practice. That is, the first operation information is assumed to include information regarding the operation of the first user, who is a student, practicing calligraphy. The control unit 21 of the second information processing device 20 acquires information regarding the operation of the second user. The information regarding the operation of the second user is also referred to as second operation information. It is assumed that the operation of the second user includes an operation of writing characters as a model for calligraphy. That is, the second operation information is assumed to include information regarding the operation of the second user, who is a teacher, showing a calligraphy model.

[0024] The control unit 11 may acquire, as the first operation information, an RGB image or a depth image captured by the imaging unit 14 of the first information processing device 10 of a first user. The control unit 21 may acquire, as the second operation information, an RGB image or a depth image captured by the imaging unit 24 of the second information processing device 20 of a second user. The control unit 11 or 21 may acquire, as the second operation information, an RGB image or a depth image from an external imaging device or distance measuring device via the interface 13 or 23.

[0025] <Model Generation> The control unit 11 of the first information processing device 10 generates a first model linked to the action of the first user based on the first action information. The control unit 11 may extract skeletal information of the first user from the first action information and generate the first model based on the skeletal information. The control unit 11 may generate the first model based on the skeletal information of the first user and an RGB image.

[0026] The control unit 21 of the second information processing device 20 generates a second model linked to the action of the second user based on the second action information. The control unit 21 may extract skeletal information of the second user from the second action information and generate the second model based on the skeletal information. The control unit 21 may generate the second model based on the skeletal information of the second user and an RGB image.

[0027] The control unit 11 or 21 may generate the first model or the second model as a 3D model.

[0028] The skeletal information may include information representing the positions of joints corresponding to human joints. The skeletal information may include information representing the positions and directions of bones connecting the joints. The control unit 11 may extract the skeletal information of the first user based on a depth image of the first user. The control unit 21 may extract the skeletal information of the second user based on a depth image of the second user.

[0029] <View model> The control unit 11 of the first information processing apparatus 10 superimposes the first model and the second model and displays them on the display device as the interface 13. The control unit 11 may acquire the second model from the second information processing apparatus 20. The control unit 11 may acquire the second operation information from the second information processing apparatus 20 and generate the second model. The control unit 21 of the second information processing apparatus 20 may superimpose the first model and the second model and display them on the display device as the interface 13. The control unit 21 may acquire the first model from the first information processing apparatus 10. The control unit 21 may acquire the first operation information from the first information processing apparatus 10 and generate the first model.

[0030] The first user practices calligraphy while looking at the display in which the first model and the second model are superimposed. For example, as shown in FIG. 2, the user 30 corresponding to the first user who is a student may practice calligraphy while looking at the display of the display device as the interface 13. In FIG. 2, the user 30 holds the brush 32 with the hand 31 and writes characters on the paper 33 while looking at the model 34. On the other hand, assume that the second user who is an instructor is writing characters in order to show the calligraphy model in another place.

[0031] The control unit 11 acquires the first operation information regarding the user 30 as the first user, generates the first model, and displays the display image 40 of the first model on the interface 13. The display image 40 of the first model includes a display image 41 corresponding to the hand 31 of the user 30 and a display image 42 corresponding to the brush 32. The control unit 11 may arrange the first model in accordance with the reference position set in the virtual space and display the display image 40 as an image viewed from a predetermined viewpoint of the virtual space.

[0032] On the one hand, the control unit 21 acquires second operation information regarding the second user, generates a second model, and outputs it to the first information processing apparatus 10. The control unit 11 acquires the second model from the second information processing apparatus 20 and displays the display image 50 of the second model on the interface 13. The display image 50 of the second model includes a display image 51 corresponding to the hand of the second user and a display image 52 corresponding to the pen. The control unit 11 may arrange the second model in accordance with a reference position set in the virtual space and display the display image 50 as an image viewed from a predetermined viewpoint of the virtual space.

[0033] The control unit 11 may display the display image 40 of the first model or the display image 50 of the second model on the interface 13 as a stereoscopic image synthesized from an RGB image and a depth image, or may display it only with the RGB image. The control unit 11 may generate polygons of the first model or the second model based on the depth image and generate a stereoscopic image by rendering the RGB image onto the polygons.

[0034] In FIG. 2, the control unit 11 displays each display image on the interface 13 as a mirror image (an image reversed left and right). By doing so, it becomes easier for the first user to imitate the actions of the second user. The control unit 11 may display each display image as an image that is not reversed.

[0035] Here, when the display image 40 of the first user and the display image 50 of the second user are displayed completely overlapping, it is difficult for the first user to visually recognize the actions of the second user. Therefore, the control unit 11 may make the reference position that is the origin of the coordinate system when arranging the first model in the virtual space and the reference position that is the origin of the coordinate system when arranging the second model in the virtual space different from each other. The reference position that is the origin of the coordinate system when arranging the first model in the virtual space is also referred to as the first reference position. The reference position that is the origin of the coordinate system when arranging the second model in the virtual space is also referred to as the second reference position. That is, the control unit 11 may make the first reference position and the second reference position different from each other. Due to the reference positions being different from each other, the display image 40 of the first model and the display image 50 of the second model may be displayed shifted on the interface 13.

[0036] The control unit 11 displays the hands of the first user and the second user so that they are slightly offset and do not completely overlap, making it easier for the first user to see the demonstration operation of the second user. For example, the control unit 11 may make the first reference position and the second reference position different while the second user is performing the demonstration operation or while the first user is performing a practice operation imitating the demonstration. In other words, the control unit 11 may make the first reference position and the second reference position different while the first user or the second user is continuing a predetermined operation. The predetermined operation may include the operation of a calligraphy demonstration by the second user. The predetermined operation may include the operation of calligraphy practice by the first user.

[0037] The control unit 11 may determine based on the first operation information whether the first user is continuing a predetermined operation. For example, the control unit 11 may determine whether the first user is continuing a predetermined operation by comparing the movement of the joints or bones of the first user's skeletal information with the movement of the joints or bones in the predetermined operation. The control unit 11 may also determine whether the first user is continuing a predetermined operation based on an image of the first user. The control unit 11 may determine whether the first user is continuing a predetermined operation by comparing the image of the first user with an image of the predetermined operation. When information such as characters is recorded on a medium such as paper by the first user, the control unit 11 may determine that the first user is continuing a predetermined operation. When a writing instrument such as a pen held by the first user is in contact with a medium such as paper, the control unit 11 may determine that the first user is continuing a predetermined operation.

[0038] The control unit 11 may acquire second operation information from the second information processing device 20 and determine based on the second operation information whether the second user is continuing a predetermined operation. The control unit 11 may determine whether the second user is continuing a predetermined operation based on an image of the second user. The control unit 11 may make a determination as to whether the second user is continuing a predetermined operation in the same manner as the determination regarding the first user. The control unit 11 may also acquire from the second information processing device 20 a determination result as to whether the second user is continuing a predetermined operation. In this case, the control unit 21 may determine based on the second operation information whether the second user is continuing a predetermined operation. The control unit 21 may make a determination as to whether the second user is continuing a predetermined operation in the same manner as the control unit 11.

[0039] The control unit 11 may make the first reference position and the second reference position different so that the parts that are characteristic of the predetermined operation are less likely to overlap. For example, in the case where the movement of the right hand holding a writing brush in a calligraphy operation is characteristic, the second reference position may be set so that the displayed image 51 of the right hand of the second user moves to the right side on the screen compared to the displayed image 41 of the right hand of the first user. Conversely, when the displayed image 51 of the right hand of the second user moves to the left side, the displayed image 51 of the right hand of the second user may be hidden by the torso part of the displayed image 40 of the first user. By specifying the direction in which the reference position is moved, the displayed image 51 of the right hand of the second user is less likely to be hidden by the displayed image 40 of the first user.

[0040] As described above, during the period when the first user or the second user is continuing a predetermined operation, by making the first reference position and the second reference position different, the visibility of the operation of the second user by the first user can be improved.

[0041] When the first user and the second user are not continuously performing a predetermined operation, the control unit 11 may set the first reference position and the second reference position to the same position. By doing so, it becomes easier for the first user to align their own posture with the posture of the second user. When the display image 50 of the second user is hidden by the display image 40 of the first user, the control unit 11 may save the operation of the display image 50 of the second user so that it can be reproduced and displayed later. When the display image 40 of the first user is hidden by the display image 50 of the second user, the control unit 11 may save the operation of the display image 40 of the first user so that it can be reproduced and displayed later. By doing so, each user can later visually recognize the differences from the operations of the other user.

[0042] The control unit 11 may acquire information regarding the characters written by the second user on paper and display it as a display image 53 on the interface 13. The characters written by the second user on paper are obtained as a result of the operation of the second user. The information regarding the characters written by the second user on paper is also referred to as the operation result of the second user. The operation of the second user writing characters on paper can be rephrased as an operation of recording information such as characters on a medium such as paper. When the predetermined operation is an operation of recording information on a medium, the control unit 11 may display the information (characters written on paper) recorded on the medium by the second user on the interface 13 so that the first user can visually recognize it. By doing so, the first user can confirm the characters written by the second user who is the instructor. As illustrated in FIG. 2, the control unit 11 may display the characters as the operation result of the second user so as to correspond to the actual location where the second user is operating. The control unit 11 may display the characters as the operation result of the second user on the interface 13 in a manner of viewing from directly in front. The control unit 11 may also display or project an image of the characters as the operation result of the second user onto the paper 33 on which the first user is writing characters. The control unit 11 may display the characters as the operation result of the second user on the interface 13 in various manners.

[0043] <Example of the procedure of the information processing method> As described above, in the information processing system 1 according to the present embodiment, the control unit 11 of the first information processing device 10 superimposes and displays the operation of the second user on the operation of the first user so that the first user can imitate and practice the actions of the second user. The control unit 11 of the first information processing device 10 may execute an information processing method including the procedures of the flowchart illustrated in FIG. 3, for example. The information processing method may be realized as an information processing program to be executed by the control unit 11. The information processing program may be stored in a non-transitory computer-readable medium.

[0044] The control unit 11 acquires first operation information (step S1). The control unit 11 generates a first model based on the first operation information (step S2). The control unit 11 acquires a second model from the second information processing device 20 (step S3). The control unit 11 determines whether the first user or the second user is continuing a predetermined operation (step S4).

[0045] When the first user or the second user is continuing a predetermined operation (step S4: YES), the control unit 11 displays the first model and the second model at different reference positions (step S5). The control unit 11 displays the operation result obtained by the second user continuing the predetermined operation (step S6). After the procedure of step S6 ends, the control unit 11 ends the execution of the procedures of the flowchart in FIG. 3.

[0046] When the first user or the second user is not continuing a predetermined operation (step S4: NO), the control unit 11 displays the first model and the second model at the same reference position (step S7). After the procedure of step S7 ends, the control unit 11 ends the execution of the procedures of the flowchart in FIG. 3.

[0047] By repeatedly executing the procedure of the flowchart in FIG. 3, the control unit 11 may continuously display the operations of the first model and the second model. In the procedure of step S3, the control unit 11 may acquire the second operation information from the second information processing device 20 and generate the second model. In the determination procedure of step S4, the control unit 11 may acquire from the second information processing device 20 the determination result as to whether the second user is continuing a predetermined operation. After the display procedure of step S7, the control unit 11 may save the operations of the display image 40 of the first user or the display image 50 of the second user so that they can be reproduced and displayed later.

[0048] <Parentheses> As described above, according to the information processing system 1 according to the present embodiment, while the first user or the second user continues a predetermined operation, the first user and the second user are displayed at different reference positions from each other. By doing so, the visibility of the operation of the second user when the first user practices a predetermined operation can be improved. As a result, the practice efficiency of the first user can be improved.

[0049] (Other Embodiments) Hereinafter, a configuration example of the information processing system 1 according to other embodiments will be described.

[0050] While the first user or the second user continues a predetermined operation, the control unit 11 of the first information processing device 10 may display the transmittance of one of the display image 40 of the first model or the display image 50 of the second model to be equal to or higher than a predetermined value. For example, the control unit 11 may set the transmittance of the display image 40 of the first model to be equal to or higher than a predetermined value and superimpose and display it on the display image 50 of the second model. By doing so, the first user can easily confirm both the operation of the first user himself / herself and the operation of the second user. That is, the visibility of the operation of the second user when the first user practices a predetermined operation can be improved. While the second user is performing a demonstration operation, the control unit 11 may set the transmittance of the display image 50 of the second model to be equal to or higher than a predetermined value and superimpose and display it on the display image 40 of the first model.

[0051] As described above, the information processing system 1 may be used not only for calligraphy practice but also for practicing various actions such as dance movements, pottery-making movements, or cooking movements. When the information processing system 1 is used for practicing dance movements, the predetermined action may correspond to the movement of each part of the body in dance or the movement of the props used in dance. When the information processing system 1 is used for practicing pottery-making movements, the predetermined action may correspond to the movement of the hands in pottery-making. The action result may correspond to the result of shaping the clay. When the information processing system 1 is used for practicing cooking movements, the predetermined action corresponds to the movement of the hands in cooking or the movement of tools such as a kitchen knife. The action result may correspond to the state of the processed food ingredients.

[0052] The information processing system 1 may cause one of the first information processing device 10 or the second information processing device 20 to function as a host server and the other to function as a client server.

[0053] Although the embodiments according to the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means or each step etc. can be rearranged so as not to be logically contradictory, and a plurality of means or steps etc. can be combined into one or divided.

Explanation of Reference Numerals

[0054] 1 Information processing system 10 First information processing device (11: Control unit, 12: Storage unit, 13: Interface, 14: Photographing unit) 20 Second information processing device (21: Control unit, 22: Storage unit, 23: Interface, 24: Photographing unit) 30 User (31: Hand, 32: Brush, 33: Paper, 34: Model) 40 - 42, 50 - 53 Display images

Claims

1. An information processing method executed by an information processing apparatus, comprising: obtaining a first model linked to the operation of a first user; obtaining a second model linked to the operation of a second user; displaying a virtual space in which the first model and the second model are arranged at different reference positions so as to be displaced while overlapping each other while the first user or the second user is continuously performing a predetermined operation; An information processing method comprising the above.

2. The information processing method according to claim 1, further comprising at least one of determining whether the first user is continuously performing the predetermined operation based on an image of the first user, or determining whether the second user is continuously performing the predetermined operation based on an image of the second user.

3. The information processing method according to claim 1 or 2, further comprising, when the predetermined operation is an operation of recording information on a medium, displaying the information recorded on the medium by the second user so that the first user can visually recognize it.

4. An information processing method executed by an information processing apparatus, comprising: obtaining a first model linked to the operation of a first user; obtaining a second model linked to the operation of a second user; displaying the first model and the second model at different reference positions while the first user or the second user is continuously performing a predetermined operation; displaying the first model and the second model at the same reference position when the first user and the second user are not continuously performing the predetermined operation; storing the operations of the first model or the second model while the first user and the second user are not continuously performing the predetermined operation so that they can be reproduced and displayed; An information processing method comprising the above.

5. An information processing apparatus comprising a control unit, wherein the control unit obtains a first model linked to the operation of a first user, obtains a second model linked to the operation of a second user, and displays a virtual space in which the first model and the second model are arranged at different reference positions so as to be displaced while overlapping each other while the first user or the second user is continuously performing a predetermined operation.

6. An information processing apparatus comprising a control unit, wherein the control unit obtains a first model linked to the operation of a first user, obtains a second model linked to the operation of a second user, While the first user or the second user is continuing a predetermined operation, display the first model and the second model at different reference positions from each other. When the first user and the second user are not continuing the predetermined operation, display the first model and the second model at the same reference position. An information processing apparatus that stores so that the operation of the first model or the second model can be reproduced and displayed while the first user and the second user are not continuing the predetermined operation.

Images