Mixed reality systems, educational support methods and programs

Abstract

We provide a mixed reality system and other technologies suitable for efficiently managing the number of people needed as instructors. [Solution] The mixed reality system 1 supports, for example, nursing education. The navigation display unit 39 of the control device 3 instructs the first display device 11 worn by the first participant 27 to perform actions for measurement. The measurement detection unit 41 detects the actions performed by the first participant 27 to perform measurements on the first biological model 5 using mixed reality technology and outputs vital data to the first display device 11. The test processing unit 45 displays a test on the first display device 11 corresponding to the output vital data and displays an explanation corresponding to the answer entered by the first participant 27.

Description

Technical Field

[0001] The present invention relates to a mixed reality system, an educational support method, and a program, and particularly to a mixed reality system that supports education related to the human body and the like.

Background Art

[0002] Patent Document 1 describes a riding simulation device that can educate the behavior of a two-wheeled vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described in Patent Document 1, in a simulation device, evaluation is performed according to the operation situation grasped in the device. In a simulation device, usually, for example, the inclination angle of a two-wheeled vehicle becomes a problem, and handling the two-wheeled vehicle with bare hands is not grasped.

[0005] However, in services for the human body, for example, the impressions felt by the other party are different when touching with the hand and when touching with the foot. Also, correct measurement results cannot be obtained unless a measuring device is properly worn. Such matters could not be evaluated by a simulation device or the like. Therefore, in practice, the instructor always had to grasp the situation, which imposed a great burden on the instructor, and it was difficult to improve the efficiency of the number of people required for guidance.

[0006] Therefore, an object of the present invention is to provide a mixed reality system and the like suitable for reducing the burden on the instructor.

Means for Solving the Problems

[0007] The first aspect of the present invention is a mixed reality system for supporting education about living organisms, comprising a first display device worn by a first participant, a control device, a first biological model, and a first measuring device, wherein the control device comprises a navigation display unit, a measurement detection unit, a vital data display unit, and a test processing unit, the navigation display unit displays instructions on the first display device for the first participant to take a measurement action, and the measurement detection unit detects, based on the positions of the first participant and the first biological model as grasped by mixed reality technology, that the first participant has touched a first detection position of the first biological model, and then... The vital data display unit outputs vital data corresponding to the first detection position to the first display device. The measurement detection unit, based on the positions of the first measuring device and the first biological model as determined by mixed reality technology, detects that the first participant has attached the first measuring device to the second detection position of the first biological model. In this case, the vital data display unit outputs vital data corresponding to the second detection position to the first display device. The test processing unit displays a test on the first display device corresponding to the output vital data and displays an explanation corresponding to the answer entered by the first participant.

[0008] A second aspect of the present invention is the mixed reality system of the first aspect, the mixed reality system supporting nursing education, the control device comprising a bio-change output unit, the first bio-model being a model that mimics a patient, the first bio-model comprising a posture changing device, the posture changing device changing the posture of the first bio-model in response to the operation of the first participant, and the bio-change output unit changing the sound and / or image output to the first display device in accordance with the posture of the first bio-model as grasped by mixed reality technology.

[0009] A third aspect of the present invention is a mixed reality system of the first or second aspect, wherein a test setting unit in the control device is set with vital data necessary to be measured in order to answer a test displayed by the test processing unit, the test processing unit displays the test on the first display device after all vital data corresponding to the detection positions set in the test setting unit have been output, and the navigation display unit displays on the first display device at an intermediate time before the test processing unit displays the test to instruct the first participant to take an action to output vital data that has been set in the test setting unit but has not yet been output.

[0010] A fourth aspect of the present invention is a mixed reality system according to any of the first to third aspects, the mixed reality system comprising a second display device worn by a second participant and a second bio-model, the second participant performing operations on the second bio-model, and the navigation display unit, in response to instructions from an instructor, displays instructions to the first display device and the second display device for the actions to be taken by the first participant and the second participant.

[0011] A fifth aspect of the present invention is an educational support method for supporting education related to living organisms using a mixed reality system, wherein the mixed reality system comprises a first display device worn by a first participant, a control device, a first biological model, and a first measuring device, the control device comprising a navigation display unit, a measurement detection unit, a vital data display unit, and a test processing unit, the navigation display unit displays instructions on the first display device for the first participant to take a measurement action, and the measurement detection unit detects the first participant touching a first detection position of the first biological model based on the positions of the first participant and the first biological model as grasped by mixed reality technology. If this is detected, the vital data display unit outputs vital data corresponding to the first detection position to the first display device; if the position of the first measuring device and the first biological model is grasped by augmented reality technology, the vital data display unit detects that the first participant has attached the first measuring device to the second detection position of the first biological model, the vital data display unit outputs vital data corresponding to the second detection position to the first display device; and the test processing unit displays a test on the first display device corresponding to the output vital data and displays an explanation corresponding to the answer entered by the first participant.

[0012] A sixth aspect of the present invention is an educational support method of the fifth aspect, wherein the mixed reality system supports nursing education, the control device comprises a bio-change output unit, the first bio-model is a model that mimics a patient, the first bio-model comprises a posture changing device, the posture changing device changes the posture of the first bio-model in accordance with the operation of the first participant, and the bio-change output unit changes the sound and / or image output to the first display device in accordance with the posture of the first bio-model as grasped by mixed reality technology.

[0013] A seventh aspect of the present invention is an educational support method of the fifth or sixth aspect, wherein the test setting unit of the control device is set with vital data necessary to be measured in order to answer the test displayed by the test processing unit, the test processing unit displays the test on the first display device after all the vital data corresponding to the detection positions set in the test setting unit has been output, and the navigation display unit displays on the first display device at an intermediate time before the test processing unit displays the test, instructing the first participant to take action to output vital data that has been set in the test setting unit but has not yet been output.

[0014] The eighth aspect of the present invention is an educational support method for any of the fifth to seventh aspects, wherein the mixed reality system comprises a second display device worn by a second participant and a second bio-model, the second participant performs processing on the second bio-model, and the navigation display unit, in response to instructions from an instructor, displays instructions to the first display device and the second display device for the actions to be taken by the first participant and the second participant.

[0015] The ninth aspect of the present invention is a program for causing a computer to function as a control device for any of the first to fourth aspects. Alternatively, the present invention may be considered as a computer-readable recording medium for recording the program of the ninth aspect. [Effects of the Invention]

[0016] According to each aspect of the present invention, by using mixed reality technology to grasp precise spatial relationships in the real world and evaluate the actions of participants, the burden on instructors can be greatly reduced and the number of people required for instruction can be efficiently managed. [Brief explanation of the drawing]

[0017] [Figure 1] This block diagram shows an example of the configuration of a mixed reality system 1, which is an example of an embodiment of the present invention. [Figure 2] It is a flowchart showing an example of the processing in the composite reality system 1 of FIG. 1.

Mode for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to this embodiment.

[0019] FIG. 1 is a block diagram showing an example of the configuration of a composite reality system 1 which is an example of an embodiment of the present invention. The composite reality system 1 supports a leader in providing education related to a living body to an experiencer. In this example, the case where the leader 25 provides nursing education to the first experiencer 27 and the second experiencer 29 will be described as an example.

[0020] The composite reality system 1 includes a control device 3, a first living body model 5, and a second living body model 15. Further, the composite reality system 1 includes, corresponding to the first living body model 5, a first measuring device 7, a second measuring device 9, a first display device 11, and a first wearing device 13. The composite reality system 1 includes, corresponding to the second living body model 15, a third measuring device 17, a fourth measuring device 19, a second display device 21, and a second wearing device 23.

[0021] The control device 3 includes a processing device 31, a storage device 33, and an input / output device 35.

[0022] The processing device 31 is, for example, a processor or the like and performs data processing. The processing device 31 includes a living body change output unit 37, a navigation display unit 39, a measurement detection unit 41, a vital data display unit 43, and a test processing unit 45. Each part included in the processing device 31 can be realized by, for example, a processor or the like that operates under the control of a program.

[0023] The storage device 33 is, for example, a memory or the like and stores data. The storage device 33 includes a living body change setting unit 47, an action instruction setting unit 49, and a test setting unit 51.

[0024] The input / output device 35 is, for example, a keyboard, a mouse, a touch panel, etc., and is for the instructor 25 to operate for input or output to the instructor 25. Further, the input / output device 35 includes an overall imaging device 36 that images the actions of the first subject 27 and the second subject 29.

[0025] The first biological model 5 is a model simulating a living body. In this example, it is a model simulating a patient. The first biological model 5 includes a first detection position 53, a second detection position 55, a third detection position 57, and a posture changing device 59.

[0026] The first detection position 53 is a position in the first biological model 5 that detects direct contact by the first subject 27.

[0027] The second detection position 55 and the third detection position 57 are positions in the first biological model 5 that respectively detect contact by the first subject 27 using the first measuring device 7 and the second measuring device 9.

[0028] In this example, it is assumed that the first detection position 53, the second detection position 55, and the third detection position 57 are different positions. However, the first detection position 53, the second detection position 55, and the third detection position 57 may be the same position. For example, the measurement detection unit 41 may handle the same position as the first detection position 53 if the first subject 27 touches it directly, as the second detection position 55 if the first subject 27 wears the first measuring device 7, and as the third detection position 57 if the first subject 27 wears the second measuring device 9, etc.

[0029] The posture changing device 59 is, for example, a device that changes the posture (body posture, shape) of the first biological model 5 so as to bend and extend at the joints of the first biological model 5.

[0030] Mixed reality system 1 allows the first participant 27 and the second participant 29 to experience mixed reality. Mixed reality is a technology that precisely superimposes the coordinate spaces of the real world and the virtual world, thereby combining and merging the real and virtual worlds to create a space in which they mutually influence each other in real time, allowing users to experience the real and virtual worlds simultaneously.

[0031] In the mixed reality system 1, mixed reality technology is used to precisely determine the spatial relationships in the real world. In this example, the measurement and detection unit 41 can precisely determine the external position of the first biological model 5 in the real world (position and shape perceived from the outside; the posture of the first biological model 5), the external positions of the first measuring device 7 and the second measuring device 9 (position and shape perceived from the outside), and the external position of the first participant 27 (position and shape perceived from the outside), etc., by referring to the images captured by the image capture device 63. Here, the images may be either videos or still images.

[0032] Therefore, the measurement and detection unit 41 can, for example, determine the posture of the first biological model 5, such as the state of bending and straightening of each joint of the first biological model 5, and whether it is lying on its back, stomach, or side. The posture changing device 59 may, in order to improve the accuracy of determining the posture of the first biological model 5, for example, detect the state of bending and straightening of each joint of the first biological model 5, or detect the orientation of the first biological model 5, such as lying on its back or stomach.

[0033] Furthermore, the measurement detection unit 41 can detect the state of the first measuring device 7 and the second measuring device 9, including whether they have been removed from the first biological model 5, whether they are properly attached to the first biological model 5, and, if they are properly attached to the first biological model 5, their position on the first biological model 5.

[0034] Furthermore, the measurement and detection unit 41 can detect whether the first participant 27 has directly touched the first biological model 5, and if so, the location of the touch on the first biological model 5.

[0035] The first display device 11 is, for example, a head-up display. The first participant 27 wears the first display device 11. The first display device 11 allows the first participant 27 to experience mixed reality.

[0036] The first display device 11 includes an image display device 61, an image capture device 63, a sound output device 65, a sound input device 67, and a user control device 69.

[0037] The image display device 61 is a display device such as a display. The image capturing device 63 is an image capturing device such as a camera. Here, the image may be a video or a still image.

[0038] The sound output device 65 is a device that outputs sound, such as a speaker. The sound input device 67 is a device that inputs sound, such as a microphone.

[0039] The participant control device 69 is, for example, a processor and performs data processing. The participant control device 69 includes a mixed reality processing unit 71. The mixed reality processing unit 71 is implemented by, for example, a processor operating under the control of a program and performs data processing to allow the first participant 27 to experience mixed reality.

[0040] The first participant 27 inputs necessary data, for example, to update the content displayed on the image display device 61 or to answer a test displayed on the image display device 61. The first wearable device 13 is worn by the first participant 27 on their arm or elsewhere and detects actions taken by the first participant 27 to input data into the first display device 11. The mixed reality processing unit 71 identifies the data input by the first participant 27 based on the detected actions of the first participant 27. If the data input by the first participant 27 can be identified by audio input to the sound input device 67, the first wearable device 13 can be omitted.

[0041] The second biological model 15 and its corresponding third measuring instrument 17, fourth measuring instrument 19, second display device 21, and second attachment device 23 can be implemented in the same manner as the first biological model 5 and its corresponding first measuring instrument 7, second measuring instrument 9, first display device 11, and first attachment device 13, respectively.

[0042] The second biological model 15 is a model that mimics a living organism and includes a fourth detection position 73, a fifth detection position 75, a sixth detection position 77, and a posture changing device 79. The fourth detection position 73 is a position in the second biological model 15 where it is detected when the second participant 29 directly touches it. The fifth detection position 75 and the sixth detection position 77 are positions in the second biological model 15 where it is detected when the second participant 29 touches it using the third measuring device 17 and the fourth measuring device 19, respectively. The posture changing device 79 is, for example, located at the joints of the second biological model 15 and can be bent and extended.

[0043] The measurement and detection unit 41 can precisely determine the position of the external shape of the second biological model 15 in the real world, the positions of the external shapes of the third measuring instrument 17 and the fourth measuring instrument 19, and the position of the external shape of the second participant 29 by referring to the images captured by the image acquisition device 83. Here, the images may be either video or still images.

[0044] The second display device 21 is worn by the second participant 29, allowing the second participant 29 to experience mixed reality. The second display device 21 comprises an image display device 81, an image capture device 83, a sound output device 85, a sound input device 87, and a participant control device 89. The image display device 81 and the image capture device 83 are a display device and an image capture device, respectively. The sound output device 85 and the sound input device 87 are devices that output and input sound, respectively. The participant control device 89 is, for example, a processor and includes a mixed reality processing unit 91 that performs data processing for the purpose of experiencing mixed reality.

[0045] The second wearable device 23 detects actions taken by the second participant 29 to input data into the second display device 21. However, it can be omitted if not necessary.

[0046] Figure 2 is a flowchart showing an example of the processing in the mixed reality system 1 of Figure 1. Below, we will explain the processing for the first participant 27 in Figure 2.

[0047] For simplicity, the first participant 27 will take action according to the vital signs displayed in the first biological model 5, measure the first biological model 5, and perform a test based on the measurement results. Here, the vital signs will be represented as respiration. In this invention, however, other vital signs such as pulse rate and respiratory rate may also be included. Furthermore, in this invention, for example, the action and measurement may be performed multiple times, and the test may be performed multiple times.

[0048] The instructor 25 operates the input / output device 35 to set up the system for the first participant 27 (Step STA1).

[0049] The instructor 25 sets the changes in the vital signs of the first biological model 5 in the biological change setting unit 47. For example, the instructor sets the initial posture of the first biological model 5, the respiratory sounds output according to the posture of the first biological model 5 and the images to be displayed overlaid on the abdomen, chest, etc., and the respiratory sounds output and images to be displayed overlaid on the abdomen, chest, etc., when the first biological model 5 maintains the same posture for a certain period of time.

[0050] The instructor 25 sets the items to be displayed on the first display device 11 to instruct the first participant 27 to take action, in the action instruction setting unit 49. For example, the instructor sets the personal information of the patient set on the first biological model 5 (name, age, gender, etc.), whether or not measures to stabilize the vital signs are necessary, the items to be displayed to instruct the action to take if measures to stabilize the vital signs are necessary, the posture of the first biological model 5 when the vital signs are stable, the items to be displayed as hints to give to the first participant 27 after a certain period of time has elapsed, and the items to be displayed to instruct the measurement when the vital signs are stable.

[0051] The instructor 25 sets the items to be displayed to the first participant 27 for the test on the first display device 11 in the test setting unit 51. For example, the instructor sets items that specify the test to be displayed to the first participant 27, display items that will be given to the first participant 27 as a hint after a certain amount of time has elapsed, and display items that will be displayed in response to the first participant 27's answers.

[0052] Furthermore, the instructor 25 sets the detection positions necessary to answer the test in the test setting unit 51. In the following example, it is assumed that the settings are to touch the first detection position 53 with the hand, to attach the first measuring device 7 to the second detection position 55, and to attach the second measuring device 9 to the third detection position 57. It is also assumed that the external shapes (shapes that can be perceived from the outside) of the first measuring device 7 and the second measuring device 9 when attached to the second detection position 55 and the third detection position 57 are set. Furthermore, it is assumed that the vital data displayed when the first detection position 53 is touched with the hand, the vital data displayed when the first measuring device 7 is attached to the second detection position 55, and the vital data displayed when the second measuring device 9 is attached to the third detection position 57 are set.

[0053] The instructor 25 prepares for the first participant 27. The preparations include, for example, setting the first biological model 5 to the posture corresponding to the initial value, placing the first measuring instrument 7 and the second measuring instrument 9 in their designated positions, and having the first participant 27 wear the first display device 11 and the first attachment device 13.

[0054] When the instructor 25 is ready for the first participant 27, he / she operates the input / output device 35 to instruct the first participant 27 to begin instruction. The bio-change output unit 37 waits until it receives an instruction from the instructor 25 to begin instruction (step STA2), and then proceeds to step STA3.

[0055] In step STA3, the biological change output unit 37 refers to the biological change setting unit 47 and starts the process of outputting a respiratory sound corresponding to the posture of the first biological model 5 in the sound output device 65 and outputting an image superimposed on the first biological model 5 in the image display device 61. In subsequent processing, the biological change output unit 37 refers to the biological change setting unit 47 and changes the sound or image accordingly if the posture of the first biological model 5 changes or if a certain period of time has passed without the posture changing.

[0056] The navigation display unit 39 refers to the action instruction setting unit 49 and displays the patient's personal information (name, age, gender, etc.) set in the first biological model 5 on the image display device 61.

[0057] The navigation display unit 39 refers to the action instruction setting unit 49 to determine whether or not measures are needed to stabilize the vital signs (step STA4). If measures are needed, proceed to step STA5. If measures are not needed, proceed to step STA8.

[0058] In step STA5, the navigation display unit 39 refers to the action instruction setting unit 49 and displays an instruction for the action to be taken on the image display device 61.

[0059] The first participant 27 bends and extends the posture changing device 59, and changes the orientation of the first biological model 5 to make it lie on its back, stomach, or side. The measurement detection unit 41 grasps the posture of the first biological model 5 from the images captured by the image acquisition device 63. The measurement detection unit 41 refers to the action instruction setting unit 49 and determines whether the vital signs will stabilize depending on the grasped posture of the first biological model 5 (step STA6). If the vital signs will stabilize depending on the posture of the first biological model 5, the procedure is evaluated as complete (determined as YES in step STA7), and the process proceeds to step STA8. If the vital signs will not stabilize depending on the posture of the first biological model 5, the process returns to step STA6.

[0060] In step STA8, the navigation display unit 39 refers to the test setting unit 51 to determine whether a measurement process is necessary to answer the test. If measurement is necessary, proceed to step STA9. If measurement is not necessary, proceed to step STA12.

[0061] In this example, the test setting unit 51 is configured to include touching the first detection position 53 with a hand, attaching the first measuring instrument 7 to the second detection position 55, and attaching the second measuring instrument 9 to the third detection position 57, in order to answer the test. Therefore, the judgment in step STA8 is YES.

[0062] In step STA9, the navigation display unit 39 refers to the action instruction setting unit 49 and displays instructions for the action to be taken for measurement on the image display device 61.

[0063] The first participant 27 directly touches the first biological model 5 or attaches the first measuring device 7 and the second measuring device 9 to it (step STA10). The measurement detection unit 41 uses images captured by the image acquisition device 63 to determine the posture of the first participant 27 when they touched the first biological model 5 (for example, whether they touched it with their hands or with something other than their hands (for example, their feet), the position of the first biological model 5 that the first participant 27 touched, etc.), and the state in which the first measuring device 7 and the second measuring device 9 are attached (for example, the external shape of the first measuring device 7 and the second measuring device 9 when they are attached, the position of the first biological model 5 with the first measuring device 7 and the second measuring device 9 attached, etc.).

[0064] The measurement detection unit 41 refers to the test setting unit 51 to determine whether the first participant 27 performed the measurement process corresponding to the detected position.

[0065] In this example, the measurement detection unit 41 refers to the test setting unit 51 to determine whether the first participant 27 touched the first detection position 53 with their hand. If it is determined that the first participant 27 touched the first detection position 53 with their hand, the vital data display unit 43 refers to the test setting unit 51 to display the vital data at the time the first detection position 53 was touched on the image display device 61.

[0066] Furthermore, the measurement detection unit 41 refers to the test setting unit 51 to determine whether the first participant 27 has attached the first measuring device 7 to the second detection position 55. If it is determined that the first participant 27 has attached the first measuring device 7 to the second detection position 55, the vital data display unit 43 refers to the test setting unit 51 to display the vital data on the image display device 61 for when the first measuring device 7 was attached to the second detection position 55.

[0067] Furthermore, the measurement detection unit 41 refers to the test setting unit 51 to determine whether the first participant 27 has attached the second measuring device 9 to the third detection position 57. If it is determined that the first participant 27 has attached the second measuring device 9 to the third detection position 57, the vital data display unit 43 refers to the test setting unit 51 to display the vital data on the image display device 61 for when the second measuring device 9 was attached to the third detection position 57.

[0068] To display vital data, it's possible to show monitors and other devices that don't actually exist in the learning space, enabling more advanced learning.

[0069] Participant 27 (the first test subject) enters data indicating whether or not to continue the measurement process. If Participant 27 believes that there is vital data necessary for the decision that is not displayed, they enter data indicating to continue the measurement process. If Participant 27 believes that all vital data necessary for the decision has been displayed, they enter data indicating to end the measurement process.

[0070] The measurement detection unit 41 determines whether or not to terminate the measurement process (step STA11). If it determines that the measurement is complete, it proceeds to step STA12. If it determines that the measurement process should continue, it returns to step STA10.

[0071] Specifically, if the first participant 27 inputs data indicating that they wish to continue the measurement process, the measurement detection unit 41 determines that the measurement process should continue and returns to step STA10.

[0072] Furthermore, if the first participant 27 inputs data indicating the end of the measurement process, the measurement detection unit 41 refers to the test setting unit 51 to determine whether all vital data corresponding to the detection position has been output and whether all measurement processes have been completed. If all measurement processes have been completed, the unit evaluates that the measurement is complete and proceeds to step STA12. If there are detection positions where measurement processing has not been performed, the image display device 61 displays that there are detection positions where measurement is insufficient and returns to step STA10.

[0073] In step STA12, the test processing unit 45 refers to the test setting unit 51 and displays the test for the first participant 27 on the image display device 61. The test may include, for example, questions that ask the participant to check the detection locations that needed measurement from a list of multiple detection locations, or questions that ask the participant to check the conditions that can be expected from the vital data.

[0074] The first participant 27 inputs data to answer the displayed test. The test processing unit 45 displays the results on the image display device 61 according to the first participant 27's answers (step STA13).

[0075] The test processing unit 45 determines whether the first participant 27 passed the test (step STA14). If the test processing unit 45 determines that the first participant 27 did not pass the test, it returns to step STA13. If the test processing unit 45 determines that the first participant 27 passed the test, it terminates the process.

[0076] In steps STA6, STA10, and STA13, after a certain amount of time has elapsed, the image display device 61 displays a hint for the first participant 27 to take action.

[0077] The processing for the education of the second participant (29) can be similarly implemented using Figure 2.

[0078] Traditionally, this type of educational processing has been implemented using, for example, simulation devices. For example, in the case of operating a device like a car, the simulation device can understand and evaluate the operational situation. However, in services involving living organisms, the impression a person feels differs depending on whether they are touched by hand or foot. Such impressions could not be evaluated by simulation devices. Therefore, in practice, instructors had to constantly monitor the situation, essentially resulting in one-on-one instruction.

[0079] Furthermore, it is possible to implement everything in a virtual space. However, virtual spaces typically do not take the real world into consideration. Therefore, people working in virtual spaces usually use special input devices. In services involving living organisms, it is important to obtain specific numerical data using measuring devices, and also to be able to touch them directly with hands, etc. The educational environment in a virtual space becomes a different environment from that of the real world.

[0080] According to the present invention, by utilizing augmented reality technology, the augmented reality system 1 can grasp precise spatial relationships in the real world. Therefore, the first participant 27 and the second participant 29 can act in an educational environment similar to that of the actual site.

[0081] Furthermore, the instructor 25 can grasp the individual circumstances of the first participant 27 and the second participant 29 through the data obtained from the image capture devices 63 and 83 and the sound input devices 67 and 87, and can also grasp the overall circumstances of the first participant 27 and the second participant 29 through the overall imaging device 36. As a result, instruction can be provided to the first participant 27 and the second participant 29 at the necessary time, eliminating the need for one-on-one instruction.

[0082] According to the present invention, by using mixed reality technology to grasp precise spatial relationships in the real world and evaluating the actions of the first participant 27 and the second participant 29, the burden on the instructor 25 can be greatly reduced and the number of people required for instruction can be efficiently managed. In the present invention, the navigation display unit 39 may display instructions for the actions of one or both of the first participant 27 and the second participant 29 on one or both of the image display devices 61 and 81 in response to instructions from the instructor 25 to the input / output device 35.

[0083] Furthermore, the present invention may also be realized through the cooperative processing of the control device 3 and the user control device 69. Some or all of the processing described above as being performed by the control device 3 may also be performed by the mixed reality processing unit 71 of the user control device 69. [Explanation of symbols]

[0084] 1. Mixed Reality System 3. Control device 5. First biological model 7 First measuring device 9 Second measuring device 11. First display device 13. First mounting device 15. Second Biological Model 17 Third measuring device 19 4th measuring device 21. Second display device 23. Second mounting device 25 Leader 27 First-time experiencers 29 Second-time participant 31 Processing Unit 33 Storage device 35 Input / Output Devices 36 Overall imaging device 37. Bio-change output unit 39 Navigation display unit 41 Measurement and detection unit 43. Vital Data Display Unit 45 Test Processing Unit 47. Biological Change Setting Unit 49 Action instruction setting section 51 Test Settings Section 53 First detection position 55 Second detection position 57 Third detection position 59. Posture changing device 61 Image display device 63 Image acquisition device 65 Sound output device 67 Audio Input Device 69. User control device 71 Mixed Reality Processing Unit 73. Fourth detection position 75 Fifth detection position 77. 6th detection position 79. Posture changing device 81 Image display device 83 Image acquisition device 85. Sound output device 87 Audio Input Device 89. User control device 91 Mixed Reality Processing Unit

Claims

1. A mixed reality system that supports education related to living organisms, The system comprises a first display device worn by the first participant, a control device, a first biological model, and a first measuring instrument. The control device comprises a navigation display unit, a measurement detection unit, a vital data display unit, and a test processing unit. The navigation display unit displays instructions on the first display device for the first participant to take an action for measurement. If the measurement and detection unit detects that the first participant has touched the first detection location of the first bio-model based on the positions of the first participant and the first bio-model as determined by augmented reality technology, the vital data display unit outputs vital data corresponding to the first detection location to the first display device. If the measurement detection unit detects that the first participant has attached the first measuring device to the second detection position of the first biological model, based on the positions of the first measuring device and the first biological model as determined by mixed reality technology, the vital data display unit outputs vital data corresponding to the second detection position to the first display device. The test processing unit is a mixed reality system that displays tests corresponding to the outputted vital data on the first display device and displays explanations corresponding to the answers entered by the first participant.

2. This mixed reality system is intended to support nursing education. The control device includes a biological change output unit, The first biological model described above is a model that mimics a patient, The first biological model is equipped with a posture changing device, The posture changing device changes the posture of the first biological model in response to the operation of the first participant. The mixed reality system according to claim 1, wherein the bio-change output unit changes the sound and / or image output to the first display device according to the posture of the first biological model grasped by mixed reality technology.

3. The test setting unit of the control device is configured with vital data that needs to be measured in order to answer the test displayed by the test processing unit. The test processing unit displays the test on the first display device after all vital data corresponding to the detection positions set in the test setting unit has been output. The mixed reality system according to claim 1, wherein the navigation display unit displays, at an intermediate time before the test processing unit displays the test, an instruction on the first display device to the first participant to take an action to output vital data that has been set in the test setting unit but has not yet been output.

4. The mixed reality system comprises a second display device worn by a second participant and a second bio-model, The second participant performs a process on the second biological model. The mixed reality system according to claim 1, wherein the navigation display unit, in response to instructions from the instructor, displays to the first display device and the second display device instructions for the actions of the first participant and the second participant.

5. An educational support method that uses a mixed reality system to support education related to living organisms, The aforementioned mixed reality system comprises a first display device worn by a first participant, a control device, a first bio-model, and a first measuring instrument. The control device comprises a navigation display unit, a measurement detection unit, a vital data display unit, and a test processing unit. The navigation display unit displays instructions on the first display device for the first participant to take an action to be measured. The measurement detection unit, If the position of the first participant and the first bio-model is determined by augmented reality technology, and the first participant is detected to have touched the first detection location of the first bio-model, the vital data display unit outputs vital data corresponding to the first detection location to the first display device. If the position of the first measuring device and the first biological model is determined by augmented reality technology, and the first participant is detected to have attached the first measuring device to the second detection position of the first biological model, the vital data display unit outputs vital data corresponding to the second detection position to the first display device. An educational support method comprising the steps of the test processing unit displaying a test corresponding to the outputted vital data on the first display device and displaying an explanation corresponding to the answer entered by the first participant.

6. A program for causing a computer to function as the control device described in claim 1.

Images