Apparatus, method, and program

Abstract

SOLUTION: A first display control unit configured to display a common virtual space to each of a plurality of operators, an operation acquisition unit configured to acquire an operation on an object in the virtual space from one operator, and a second display control unit configured to display one avatar indicating the one operator in the virtual space to each operator and cause the one avatar to perform a motion corresponding to the operation, the first display control unit displays, for each operator, a field of view of the virtual space viewed from an avatar corresponding to the operator.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus, a method, and a program. [Background technology]

[0002] Patent documents 1 to 3 state that "multiple users can use virtual reality devices 18 to access visual representations of asset components within a VR or AR environment, thereby experiencing the same digital environment and collaborating with each other for design, manufacturing, or diagnostic purposes" (paragraph 0171 of Patent document 1). [Prior art document] [Patent documents] [Patent Document 1] Special Publication No. 2023-510091 [Patent Document 2] JP 2019-003545 A [Patent Document 3] JP 2020-166264 A Summary of the Invention

[0003] In a first aspect of the present invention, there is provided an apparatus comprising: a first display control unit that displays a common virtual space for each of a plurality of operators; an operation acquisition unit that acquires an operation on an object in the virtual space from one operator; and a second display control unit that displays an avatar representing the one operator in the virtual space for each operator and causes the one avatar to perform an action in accordance with the operation.

[0004] In the above device, the second display control unit may display multiple avatars representing each of the multiple operators in the virtual space, and the first display control unit may display to each operator the view of the virtual space from the avatar corresponding to that operator.

[0005] In the above device, the first display control unit may, in response to an operation from an operator, display the view of the virtual space from the one avatar to each of the multiple operators other than the one operator.

[0006] In any of the above devices in which the second display control unit displays multiple avatars representing each of the multiple operators in the virtual space, the first display control unit may, in response to an operation from an operator, direct the line of sight of each of the other avatars among the multiple avatars that is different from the one avatar to an area in the virtual space that is included in the field of view when looking at the virtual space from the one avatar.

[0007] In any of the above devices in which the second display control unit displays multiple avatars representing each of the multiple operators in the virtual space, the first display control unit may cause each of the multiple operators other than the one operator to display a marker indicating an area of ​​the virtual space that is included in the field of view of the virtual space from the one avatar.

[0008] In the above device, the first display control unit may distinguishably display an area of ​​the virtual space that is included in the field of view of the one avatar when looking at the virtual space within the virtual space displayed to each of the other operators other than the one operator among the plurality of operators.

[0009] In any of the above devices in which the second display control unit displays multiple avatars representing each of the multiple operators in the virtual space, the operation acquisition unit may acquire operations on objects in the virtual space from each operator, and the second display control unit may cause the avatar corresponding to the operator who performed the operation acquired by the operation acquisition unit to perform an action corresponding to the operation.

[0010] The above device may further include a movement restriction unit that restricts the movement of another avatar, which is different from the one avatar, among the avatars of the multiple operators, to a predetermined content for the other avatar.

[0011] The above device may further include a proficiency acquisition unit that acquires the proficiency of an object operation for another operator among the plurality of operators who is different from the one operator, and a first setting unit that sets the content of the action that the other avatar can perform to the content of the action that is pre-associated with the proficiency acquired for the other operator corresponding to the other avatar.

[0012] Any of the above devices including a movement restriction unit may further include a release unit that releases the movement restriction imposed by the movement restriction unit in response to an operation by the one operator.

[0013] In the above-mentioned device in which the second display control unit causes an avatar corresponding to an operator who performed an operation acquired by the operation acquisition unit to perform an action corresponding to the operation, the operation acquisition unit may have a cancellation unit that cancels operations on objects in the virtual space from other operators different from the one operator among the plurality of operators in response to an operation from the one operator.

[0014] Any of the above devices may further include a proficiency acquisition unit that acquires the proficiency of each operator in operating an object, and a second setting unit that sets the operator with the highest proficiency among the plurality of operators as the one operator.

[0015] In any of the above devices, the virtual space is a three-dimensional model of a facility having at least one piece of equipment as an object, and the device may further include an equipment control unit that controls the equipment corresponding to the object in accordance with an operation on the object in the virtual space acquired by the operation acquisition unit.

[0016] Any of the above devices may further include a storage unit that stores the content of each operation acquired by the operation acquisition unit.

[0017] In a second aspect of the present invention, there is provided a method comprising: a first display control step of displaying a common virtual space for each of a plurality of operators; an operation acquisition step of acquiring an operation on an object in the virtual space from one operator; and a second display control step of displaying an avatar representing the one operator in the virtual space for each operator and causing the one avatar to perform an action in accordance with the operation.

[0018] In a third aspect of the present invention, there is provided a program that, when executed by a computer, causes the computer to function as a first display control unit that displays a common virtual space for each of a plurality of operators, an operation acquisition unit that acquires operations on objects in the virtual space from one operator, and a second display control unit that displays one avatar representing the one operator in the virtual space for each operator and causes the one avatar to perform an action in accordance with the operation.

[0019] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0020] [Figure 1] 1 shows a system 1 according to a first embodiment. [Figure 2] 1 shows the operation of the operation support device 2. [Figure 3] 1 shows a system 1A according to a second embodiment. [Figure 4] 1 shows the operation of the operation support device 2A. [Figure 5] Shows a virtual space. [Figure 6] The display contents for a new operator in the state of FIG. 5 are shown below. [Figure 7]5 and 6, the display contents for the rookie operator when the rookie avatar A2 directs his / her gaze toward the gaze area of ​​the veteran avatar A1. [Figure 8] 8 shows the contents displayed to a new operator when a view sharing operation is performed from the state of FIGS. 5 to 7. FIG. [Figure 9] 12 illustrates an example computer 1200 in which aspects of the present invention may be embodied, in whole or in part. DETAILED DESCRIPTION OF THE INVENTION

[0021] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0022] (System 1 according to the first embodiment) 1 shows a system 1 according to the first embodiment. The system 1 includes an operation assistance device 2, one or more display devices 3, and one or more devices 40.

[0023] ((Display device 3)) Each display device 3 displays various information under the control of the operation support device 2. In this embodiment, as an example, the system 1 is provided with a plurality of display devices 3.

[0024] Each display device 3 may be provided for each operator and may display information to the corresponding operator. As an example, each display device 3 may be a head-mounted display, but may also be another type of display device.

[0025] ((equipment 40)) The device 40 may be an instrument, a machine, or a device. For example, the device 40 may be an actuator such as a valve, a pump, a heater, a fan, a motor, or a switch that controls at least one physical quantity such as pressure, temperature, pH, speed, or flow rate, or a controller that controls the actuator. The device may also be a sensor or an instrument that measures a condition. The device may also be a structure such as a pipe, a tank, a heat exchanger, a furnace, or a distillation column.

[0026] Each device 40 may be displayed as an object in a virtual space by the operation support device 2 described later. In the present embodiment, as an example, each device 40 may be provided in a facility 4 in the real space, and may be displayed as an object in a virtual space in which the facility 4 is three-dimensionally modeled by the operation support device 2.

[0027] The facility 4 may have at least one device 40 as an object, and may be, for example, a plant or a composite device that combines multiple devices. Examples of plants include industrial plants such as chemical and bio plants, plants that manage and control wellheads and surrounding areas of gas and oil fields, plants that manage and control power generation such as hydroelectric, thermal, and nuclear power, plants that manage and control environmental power generation such as solar and wind power, and plants that manage and control water supply and sewage systems, dams, etc.

[0028] ((Operation support device 2)) The operation assistance device 2 assists in training to operate the device 40. The operation assistance device 2 includes an operation acquisition unit 201, a device control unit 202, a display control unit 203, a storage unit 204, a skill level acquisition unit 205, and a role setting unit 206.

[0029] (((operation acquisition unit 201))) The operation acquisition unit 201 acquires various operations from multiple operators. The operation acquisition unit 201 may acquire the operations via an input device such as a keyboard or a mouse, or may acquire the operations from the movements of the operators via a conventionally known motion capture system.

[0030] The operation acquisition unit 201 may acquire an operation for each operator. The operation acquisition unit 201 according to this embodiment may acquire an operation (also referred to as an operation instruction operation) for causing an avatar displayed in a virtual space to perform an action from one of the multiple operators.

[0031] The action instruction operation may include an operation on an object in the virtual space, that is, an operation (also referred to as an object operation) that causes an avatar to perform an action on an object in the virtual space. The action instruction operation may include, in addition to an object operation, other operations that cause the avatar to perform an action such as moving or changing its orientation. When the action acquisition unit 201 acquires an operation via a motion capture system, the action instruction operation may be a gesture that imitates an action performed by an operator in real space. In this embodiment, as an example, one operator from whom the action instruction operation is acquired may be a more experienced operator (also referred to as a veteran operator) than the other operators. The operation acquisition unit 201 may supply an operation signal corresponding to the action instruction operation to the device control unit 202, the display control unit 203, and the storage unit 204.

[0032] (((Device control unit 202))) The device control unit 202 controls the device 40 corresponding to the object to be operated, in accordance with the object operation acquired by the operation acquisition unit 201. The device control unit 202 may control the device 40 by supplying a control signal in accordance with the object operation to the device 40. As an example, the device control unit 202 may drive the device 40 as an actuator.

[0033] (((display control unit 203))) The display control unit 203 performs display for each of the multiple operators via each display device 3. The display control unit 203 includes a virtual space display control unit 2031 and an avatar display control unit 2032.

[0034] ((((Virtual space display control unit 2031)))) The virtual space display control unit 2031 is an example of a first display control unit, and displays a common virtual space for each of multiple operators. The virtual space display control unit 2031 may display a virtual space in which the equipment 4 is a three-dimensional model. If a display device 3 is provided for each operator, the virtual space display control unit 2031 may display a virtual space individually for each operator. The virtual space display control unit 2031 according to this embodiment may display the same area in the virtual space for each operator.

[0035] ((((Avatar display control unit 2032)))) The avatar display control unit 2032 is an example of a second display control unit, and displays one avatar (also referred to as a veteran avatar) representing a veteran operator for each operator in the virtual space. The avatar may show the operator's entire body, or may show a part of the operator's body (for example, a hand). Displaying an avatar for each operator may mean arranging the avatar in the virtual space in a state where it is visible to each operator, or may mean displaying at least a part of the avatar (for example, a hand) within the field of view in the virtual space.

[0036] The avatar display control unit 2032 may cause the veteran avatar to perform an action in response to an action instruction operation performed by the veteran operator. For example, the avatar display control unit 2032 may cause the veteran avatar to perform an action in response to an object operation performed by the veteran operator. The action in response to the object operation may be an action instructed by the object operation, among actions performed on an object in virtual space. The action in response to the object operation may imitate an action performed by the operator on the device 40 in real space, and as an example, may imitate an action performed by the operator operating the device 40 as an actuator in real space. The action in response to the object operation may indicate an indication value of the amount of operation given to the object as an actuator, depending on the number, magnitude, and duration of the action.

[0037] (((storage unit 204))) The storage unit 204 stores various information. For example, the storage unit 204 may store the content of each object operation acquired by the operation acquisition unit 201. The content of the object operation may include identification information (also referred to as operator ID) of the operator who performed the object operation, identification information of the target object, the type of operation, and the time when the object operation was performed. The storage unit 204 may further store, for each operator, the time the corresponding avatar stayed in the virtual space and the number of object operations. The time stayed in the virtual space and the number of object operations may be calculated by the virtual space display control unit 2031, for example.

[0038] (((Proficiency level acquisition part 205))) The proficiency level acquisition unit 205 acquires the proficiency level of each operator in object manipulation. For example, the proficiency level acquisition unit 205 may acquire the proficiency level of each operator from the contents stored in the storage unit 204. As an example, the proficiency level acquisition unit 205 may calculate the proficiency level of each operator based on at least one of the number of object manipulations or the time spent in the virtual space, which are stored in the storage unit 204 for each operator. The proficiency level acquisition unit 205 may calculate a higher proficiency level the more the number of object manipulations, or the longer the time spent in the virtual space. The proficiency level acquisition unit 205 may supply the proficiency level acquired for each operator to the role setting unit 206.

[0039] (((role setting unit 206))) The role setting unit 206 is an example of a second setting unit, and sets, among the multiple operators, an operator with the highest proficiency level acquired by the proficiency level acquisition unit 205 as a veteran operator. In addition, the role setting unit 206 may reset another operator as a veteran operator in response to an operation by the veteran operator. The role setting unit 206 may supply the operator ID of the veteran operator to the operation acquisition unit 201 and the avatar display control unit 2032. This may enable the operation acquisition unit 201 to acquire an action instruction operation from the veteran operator. Furthermore, the avatar display control unit 2032 may display a veteran avatar representing the veteran operator for each operator, and cause the veteran avatar to perform an action in response to an action instruction operation by the veteran operator.

[0040] According to the operation support device 2, a veteran avatar representing a veteran operator is displayed to each operator in the virtual space, and when an object operation on an object in the virtual space is acquired from the veteran operator, the veteran avatar performs an action corresponding to the object operation. Therefore, the action performed by the veteran operator on an object in the virtual space can be shown to other operators via the veteran avatar, and information such as the content and results of the action can be shared.

[0041] Furthermore, in response to the object operation, the equipment 40 in the real space corresponding to the object to be operated is controlled. Therefore, the operation performed by the veteran avatar on the object in the virtual space where the equipment 40 in the real space can be controlled can be shown to other operators, and the information can be shared.

[0042] Furthermore, the operator with the highest level of skill among a plurality of operators is set as the veteran operator, and therefore the actions performed by the avatar of the highly skilled operator can be shown to other operators, and this information can be shared.

[0043] Furthermore, since the contents of each object operation are stored, it is possible to examine whether the object operation is appropriate or not based on the stored contents.

[0044] ((Operation of operation support device 2)) 2 shows the operation of the operation assistance device 2. The operation assistance device 2 assists in training in operating the device 40 by performing the processes of steps S11 to S23.

[0045] In step S11, the skill level acquiring unit 205 acquires the skill level of each operator in operating an object. The skill level acquiring unit 205 may acquire the skill level of each operator from the contents stored in the storage unit 204.

[0046] In step S13, the role setting unit 206 sets the operator with the highest level of proficiency acquired in step S11 as a veteran operator from among the multiple operators. The role setting unit 206 may further set each of the remaining operators as a rookie operator.

[0047] In step S15, the virtual space display control unit 2031 displays a common virtual space for each of the multiple operators. In the present embodiment, as an example, the virtual space display control unit 2031 may display for each operator the view seen from a veteran avatar placed in the virtual space.

[0048] In step S17, the avatar display control unit 2032 causes each operator to display a veteran avatar in the virtual space.

[0049] In step S19, the operation acquisition unit 201 acquires an action instruction operation from the experienced operator. Note that in this figure, a case will be described in which an object operation for an object in a virtual space is acquired as the action instruction operation.

[0050] In step S21, the avatar display control unit 2032 causes the veteran avatar to perform an action corresponding to the object operation acquired in step S19. As a result, the action performed by the veteran operator in the virtual space via the veteran avatar is displayed to each operator. Note that if an action instruction operation is performed in step S19 to cause the avatar to perform an action such as moving or changing its orientation, rather than an object operation, the avatar display control unit 2032 may cause the veteran avatar to perform an action corresponding to that operation. In this case, the virtual space display control unit 2031 may change the displayed virtual space in accordance with a change in the field of view of the veteran avatar.

[0051] In step S23, the device control unit 202 controls the device 40 corresponding to the object to be operated, in accordance with the object operation acquired in step S 19. When step S23 ends, the process may proceed to step S19 described above.

[0052] (System 1 according to the second embodiment) 3 shows a system 1A according to a second embodiment. The system 1A includes an operation assistance device 2A. In the system 1A according to this embodiment, components that are substantially the same as those in the system 1 shown in FIG. 1 are designated by the same reference numerals, and descriptions thereof will be omitted.

[0053] The operation assistance device 2A includes an operation acquisition unit 201A, a display control unit 203A, a restriction setting unit 207A, an operation restriction unit 208A, and a release unit 209A. Note that, as will be described in detail later, in the operation assistance device 2A according to this embodiment, avatars of multiple operators are displayed in a virtual space.

[0054] (((operation acquisition unit 201A))) The operation acquisition unit 201A acquires various operations from a plurality of operators in the same manner as the operation acquisition unit 201 in the first embodiment. For example, the operation acquisition unit 201A may acquire, from a veteran operator, an operation instruction for causing a veteran avatar in a virtual space to perform an action.

[0055] In addition, the operation acquisition unit 201A according to this embodiment may acquire, from another operator (also called a new operator) among the multiple operators who is different from the veteran operator, an action instruction operation to cause another avatar (also called a new avatar) representing the new operator to perform an action in the virtual space.

[0056] That is, the operation acquisition unit 201A according to this embodiment may acquire an operation instruction operation (for example, an object operation) from each operator. The operation acquisition unit 201A may supply an operation signal corresponding to the operation instruction operation to the device control unit 202, the display control unit 203, the operation restriction unit 208A, and the storage unit 204 in association with the operator ID of the operator who performed the operation.

[0057] (((display control unit 203A))) The display control unit 203A individually displays information for each of the multiple operators via each display device 3. The display control unit 203A includes an avatar display control unit 2032A and a virtual space display control unit 2031A.

[0058] ((((avatar display control unit 2032A)))) The avatar display control unit 2032A is an example of a second display control unit, and displays multiple avatars representing multiple operators in the virtual space. The avatar display control unit 2032A may display a veteran avatar representing a veteran operator in the virtual space, as well as other avatars (also referred to as rookie avatars) representing other operators (also referred to as rookie operators). Each avatar may represent the entire operator, or may represent a part of the operator's body (for example, a hand).

[0059] The avatar display control unit 2032A may cause an avatar corresponding to an operator who performed an action instruction operation (for example, an object operation) acquired by the operation acquisition unit 201 to perform an action in accordance with the operation. For example, when a new operator performs an object operation, the avatar display control unit 2032A may cause a new avatar corresponding to the new operator to perform an action in accordance with the object operation. When an operation to change the position or posture of any of the avatars is acquired via the operation acquisition unit 201A, the avatar display control unit 2032A may change the position or posture of the avatar in accordance with the operation.

[0060] ((((virtual space display control unit 2031A)))) The virtual space display control unit 2031A is an example of a first display control unit, and similar to the virtual space display control unit 2031 in the first embodiment, displays a common virtual space for each of a plurality of operators. The virtual space display control unit 2031A according to this embodiment may display, for each operator, a view of the virtual space as seen from the avatar corresponding to that operator. The virtual space display control unit 2031A may display the same area in the virtual space for each operator, or may display different areas.

[0061] The virtual space display control unit 2031A may cause each rookie operator to display a mark indicating an area of ​​the virtual space that is included in the field of view of the veteran avatar when looking at the virtual space (also referred to as a gaze area). The virtual space display control unit 2031A may cause each rookie operator to display a mark in response to the rookie avatar's field of view not including the veteran avatar's gaze area. The mark may be an arrow indicating the direction of the gaze area, and may include a message encouraging the rookie operator to direct their gaze toward the gaze area.

[0062] The virtual space display control unit 2031A may distinguishably display the veteran avatar's gaze area in the virtual space displayed to each rookie operator. The virtual space display control unit 2031A may distinguishably display the veteran avatar's gaze area in response to the rookie avatar's field of view including the veteran avatar's gaze area. As an example, the virtual space display control unit 2031A may distinguishably display the veteran avatar's gaze area by surrounding it with a frame. However, the manner of distinguishing display is not limited to this. For example, the virtual space display control unit 2031A may distinguishably display the veteran avatar's gaze area by changing the brightness of the veteran avatar's gaze area or areas other than the gaze area.

[0063] The virtual space display control unit 2031A may display to each rookie operator the view of the virtual space as seen from the veteran avatar in response to an operation from an operator (also referred to as a view sharing operation). When the view of the virtual space as seen from the veteran avatar is displayed to each rookie operator, the virtual space display control unit 2031A may display to each rookie operator a message indicating that the view as seen from the veteran avatar is being displayed. The view sharing operation may be performed by either the veteran operator or the rookie operator. The view sharing operation may be input via the operation acquisition unit 201A.

[0064] The virtual space display control unit 2031A may direct the line of sight of each new avatar toward the veteran avatar's gaze area in the virtual space in response to an operation from the operator (also referred to as a gaze tracking operation). In other words, the virtual space display control unit 2031A may cause each new operator to see the veteran avatar's gaze area from the new avatar. This allows the field of view of each new avatar to move while tracking the field of view of the veteran avatar. When the line of sight of each new avatar is directed toward the veteran avatar's gaze area, the virtual space display control unit 2031A may display a message to each new operator indicating that the field of view directed toward the veteran avatar's gaze area is being displayed. The field of view tracking operation may be performed by either the veteran operator or the new operator. The field of view tracking operation may be input via the operation acquisition unit 201A.

[0065] (((Limit setting unit 207A))) The restriction setting unit 207A is an example of a first setting unit, and sets the content of actions that a new avatar can perform. The restriction setting unit 207A may set the content of actions that a new avatar can perform to the content of actions that are previously associated with the proficiency level acquired by the proficiency level acquisition unit 205 for the new operator corresponding to the new avatar. For example, the restriction setting unit 207A may have a first correspondence table that previously associates proficiency values ​​with the content of actions that can be performed, and may set the content of actions that are previously associated with the proficiency level acquired for the new operator in the first correspondence table as the content of actions that the new avatar can perform. The restriction setting unit 207A may have a second correspondence table that associates the operator ID of each operator with the content of actions that an avatar can perform, and may set the content of actions by storing the content of actions for the operator ID of one of the operators in the second correspondence table.

[0066] Here, the executable action may be an action corresponding to an action instruction operation, or may be an action corresponding to an object operation or an operation other than an object operation (for example, an action to move an avatar or an action to change the orientation of an avatar).

[0067] The content of the executable actions may be determined by the type of action, or may be determined by the type of action and a numerical range for the action. The numerical range may indicate a range of the number of times, magnitude, or duration of the action. For example, the numerical range may limit the instruction value of the operation amount given to the actuator object.

[0068] (((operation limiting unit 208A))) The movement restriction unit 208A restricts the movement of the new avatar to the content preset for the new avatar. Here, restricting the movement of the new avatar to the content preset may mean that the new avatar is allowed to perform the movement of the content preset, and is not allowed to perform any other movement.

[0069] The action limiting unit 208A according to this embodiment may limit the action of each new avatar to the action content set for that new avatar by the limit setting unit 207A. The action limiting unit 208A may perform the restriction by referring to the content set by the limit setting unit 207A (in this embodiment, as an example, the content of the second correspondence table). The action limiting unit 208A may perform the restriction so that an action content different from the action content associated in the second correspondence table is not performed for the operator ID supplied together with the operation signal from the operation acquisition unit 201A. The action limiting unit 208A may limit the action of the new avatar by controlling the avatar display control unit 2032A.

[0070] (((Release part 209A))) The release unit 209A releases the restriction on movement imposed by the movement restriction unit 208A in response to an operation (also referred to as a release operation) from the veteran operator. The release unit 209A may supply a signal instructing the release of the movement restriction to the movement restriction unit 208A. This releases the movement restriction imposed by the movement restriction unit 208A, allowing the rookie avatar to move. The release operation may be input via the operation acquisition unit 201A. Note that in this embodiment, as an example, the release operation is performed after the movement instruction operation by the rookie operator. However, the release operation may also be performed in advance before the movement instruction operation.

[0071] According to the operation support device 2A, multiple avatars representing multiple operators are displayed in the virtual space, and the view of the virtual space from the avatar corresponding to each operator is displayed for each operator. Therefore, the virtual space can be shown to each operator from a different view.

[0072] Furthermore, the view of the virtual space from the veteran avatar is displayed to each operator in response to a view sharing operation. Therefore, when a view sharing operation is performed from a state in which each operator is viewing the virtual space from their own view, the view of the veteran operator who operated the object is shared with each operator, and the veteran operator's gaze area can be reliably shown to the novice operator.

[0073] Furthermore, the line of sight of each new avatar is directed toward the gaze area of ​​the veteran avatar in response to the gaze tracking operation. Therefore, in response to the visual field tracking operation being performed from a state in which each operator is viewing the virtual space with their own field of view, each new operator's line of sight is directed toward the gaze area of ​​the veteran operator who performed the object operation, so that the new operator can reliably see the gaze area of ​​the veteran operator.

[0074] In addition, a mark indicating the gaze area of ​​the veteran avatar is displayed for each new operator. Therefore, even if each operator is looking at the virtual space from their own field of vision, each new operator can recognize the gaze area of ​​the veteran operator who performed the object operation and be prompted to look at the gaze area of ​​the veteran operator.

[0075] Furthermore, the gaze area of ​​the veteran avatar is distinctively displayed within the virtual space displayed to each new operator, thereby ensuring that each new operator is aware of the gaze area of ​​the veteran operator who performed the object operation, and strongly urging each new operator to look at the gaze area of ​​the veteran operator.

[0076] Furthermore, object operations performed on objects in the virtual space by each operator are acquired, and the avatar corresponding to the operator who performed the object operation performs an action in accordance with the object operation. Therefore, the actions performed by each operator in the virtual space via the avatar can be shown to other operators, and the information can be shared.

[0077] Furthermore, the movement of the new avatar is restricted to the content set in advance for the new avatar, thereby preventing the new avatar from moving freely.

[0078] Furthermore, the object manipulation proficiency level of the new operator is acquired, and the content of the actions that the new avatar can perform is set to the action content that is pre-associated with the acquired proficiency level of the corresponding new operator. Thus, the action content of the new avatar can be restricted according to the object manipulation proficiency level.

[0079] Furthermore, restrictions on the actions of the rookie avatar are lifted in response to a lifting operation from the veteran operator, so that the veteran operator can approve the actions of the rookie avatar on the object.

[0080] ((Operation of the operation support device 2A)) 4 shows the operation of the operation assistance device 2 A. The operation assistance device 2 A assists training in operating the device 40 by performing the processes of steps S31 to S59.

[0081] In step S31, the skill level acquiring unit 205 acquires the skill level of each operator in operating an object, in the same manner as in step S11 described above.

[0082] In step S33, the role setting unit 206 sets the operator with the highest level of proficiency obtained in step S31 as a veteran operator from among the multiple operators. The role setting unit 206 may set the remaining operators as rookie operators.

[0083] In step S34, the restriction setting unit 207A sets the content of the actions that the new avatar can perform to the content of the actions that are associated in advance with the proficiency level acquired in step S31 for the new operator corresponding to the new avatar.

[0084] In step S35, the virtual space display control unit 2031A displays a common virtual space for each of the multiple operators. In this embodiment, as an example, the virtual space display control unit 2031A may display, for each operator, a view of the virtual space as seen from an avatar corresponding to that operator, out of multiple avatars arranged in the virtual space.

[0085] In step S37, the avatar display control unit 2032A displays an avatar in the virtual space for each operator. The avatar display control unit 2032A may display an avatar arranged in the virtual space for each operator.

[0086] In step S39, the operation acquisition unit 201A acquires an action instruction operation from any operator. As an example, the operation acquisition unit 201A may acquire an object operation for an object in the virtual space.

[0087] In step S41, the avatar display control unit 2032A determines whether the operator who performed the action instruction operation is a veteran operator. If it is determined that the operator who performed the action instruction operation is a veteran operator (step S41; Yes), the process may proceed to step S43. If it is determined that the operator who performed the action instruction operation is not a veteran operator, that is, a rookie operator (step S41; No), the process may proceed to step S51.

[0088] In step S43, the avatar display control unit 2032A causes the veteran avatar to perform an action corresponding to the action instruction operation acquired in step S39. If an object operation is acquired in step S39, the avatar display control unit 2032A may cause the veteran avatar to perform an action corresponding to the object operation. As a result, the action performed by the veteran operator in the virtual space via the veteran avatar is displayed to each operator. If an action instruction operation that causes the avatar to perform an action such as moving or changing its orientation, rather than an object operation, is acquired in step S39, the avatar display control unit 2032A may cause the veteran avatar to perform an action corresponding to the operation. In this case, the virtual space display control unit 2031A may change the virtual space displayed to the veteran operator in accordance with a change in the field of view of the veteran avatar.

[0089] In step S45, the device control unit 202 determines whether the operation acquired in step S39 was an object operation. If it is determined that it was not an object operation (step S45; No), the process may proceed to the above-mentioned step S39. If it is determined that it was an object operation (step S45; Yes), the process may proceed to step S47.

[0090] In step S47, the device control unit 202 controls the device 40 corresponding to the object to be operated, in accordance with the object operation acquired in step S39. After step S47 ends, the process may proceed to step S39 described above.

[0091] In step S51, the action restriction unit 208A determines whether the action of the new avatar in response to the action instruction operation input by the new operator in step S39 is an executable action preset for the new avatar. If it is determined to be an executable action (step S51; Yes), the process may proceed to step S53, and if it is determined to be an infeasible action (step S51; No), the process may proceed to step S55.

[0092] In step S53, the avatar display control unit 2032A causes the new avatar corresponding to the new operator who has performed the action instruction operation (for example, an object operation) to perform an action in accordance with the operation. If an object operation is acquired in step S39, the avatar display control unit 2032A may cause the new avatar to perform an action in accordance with the object operation. As a result, the action performed by the new operator in the virtual space via the new avatar is displayed to each operator. If an action instruction operation that causes the avatar to perform an action such as moving or changing its orientation, rather than an object operation, is acquired in step S39, the avatar display control unit 2032A may cause the new avatar to perform an action in accordance with the operation. In this case, the virtual space display control unit 2031A may change the virtual space displayed to the new operator in accordance with a change in the field of view of the new avatar. After the processing of step S53 is completed, the processing may proceed to step S45 described above.

[0093] In step S55, the release unit 209A determines whether or not a release operation is performed by the veteran operator. If it is determined that a release operation has been performed (step S55; Yes), the process may proceed to step S57, and if it is determined that a release operation has not been performed (step S55; No), the process may proceed to step S59.

[0094] In step S57, the release unit 209A releases the restriction on the operation imposed by the operation restriction unit 208A. When the process of step S57 ends, the process may proceed to step S53 described above.

[0095] In step S59, the movement restriction unit 208A restricts the movement of the new avatar. The movement restriction unit 208A may maintain the state of the new avatar without causing the avatar display control unit 2032 to perform the movement of the new avatar in accordance with the movement instruction operation acquired in step S39. The movement restriction unit 208A may display a message indicating that the movement of the new avatar is restricted to the relevant new operator (i.e., the new operator who performed the movement instruction operation) via the avatar display control unit 2032. When the processing of step S59 is completed, the processing may proceed to the above-mentioned step S39.

[0096] ((Example)) 5 shows a virtual space. Note that this figure shows, as an example, the virtual space of facility 4 in which pipes P1 and P2, valves B1 and B2, and a flow meter S are provided as devices 40. This figure also shows, as an example, a state in which a veteran avatar A1 and a rookie avatar A2 are placed in the virtual space, and while veteran avatar A1 is performing an action on valve B2 in response to an object operation, the line of sight of rookie avatar A2 (see the white arrow in the figure) is not directed toward the gaze area of ​​veteran avatar A1 (the area including valve B2 in this figure).

[0097] Fig. 6 shows the content displayed to the rookie operator in the state of Fig. 5. When the gaze area of ​​the veteran avatar A1 is not included in the field of view of the rookie avatar A2, an arrow indicating the direction of the gaze area and a message encouraging the rookie operator to turn his / her gaze toward the gaze area (in this figure, as an example, "Look this way") may be displayed.

[0098] Fig. 7 shows the display content for the rookie operator when the rookie avatar A2 turns his / her gaze toward the gaze area of ​​the veteran avatar A1 from the states of Fig. 5 and Fig. 6. When the gaze area of ​​the veteran avatar A1 is included in the field of view of the rookie avatar A2, the gaze area may be distinguishably displayed by being surrounded by a frame.

[0099] Figure 8 shows the display content for the rookie operator when a view sharing operation is performed from the states of Figures 5 to 7. When a view sharing operation is performed, the view of veteran avatar A1 is displayed to the rookie operator, and a message to that effect (in this figure, as an example, "View sharing in progress") may be displayed.

[0100] ((Variant)) In the above embodiment, the storage unit 204 has been described as storing data used to calculate the proficiency level, such as the time spent in the virtual space and the number of object operations for each operator, but the proficiency level itself may also be stored. In this case, the proficiency level acquisition unit 205 may acquire the proficiency level of each operator from the storage unit 204. The proficiency level stored in the storage unit may be a value calculated from the time spent in the virtual space and the number of object operations, or may be a value arbitrarily set by any operator (for example, a veteran operator).

[0101] Furthermore, although the operation assistance devices 2 and 2A have been described as including the device control unit 202, the storage unit 204, the proficiency level acquisition unit 205, and the role setting unit 206, they may not include any of these. If the operation assistance devices 2 and 2A do not include the storage unit 204, the contents of the object operations acquired by the operation acquisition units 201 and 201A may be stored in an external storage device. If the operation assistance devices 2 and 2A do not include the proficiency level acquisition unit 205 and the role setting unit 206, one of the operators may be set in advance as a veteran operator.

[0102] Furthermore, although the operation assistance device 2A has been described as having the restriction setting unit 207A and the restriction removal unit 209A, it may not have either of them. If the operation assistance device 2A does not have the restriction setting unit 207A, the movement restriction unit 208A may restrict the movements of the new avatar to executable movements that have been set in advance for the new avatar.

[0103] Furthermore, although the operation assistance device 2A has been described as including the action restricting unit 208A, it may not include the action restricting unit 208A. In this case, the operation acquisition unit 201A may include a canceling unit that cancels an object operation from a rookie operator in response to an operation (also referred to as a canceling operation) from the veteran operator. This prevents the rookie avatar from freely operating on an object. Here, the canceling operation may be performed after the object operation by the rookie operator. In this case, in response to the rookie operator performing an object operation, the content of the object operation may be displayed to the veteran operator, and the canceling unit may cancel the object operation in response to the canceling operation from the veteran operator. Alternatively, the canceling operation may be performed in advance before the object operation by the rookie operator. In this case, the canceling unit may cancel the object operation in response to the object operation that causes the rookie avatar to perform an operation different from the content of the operation set by the restriction setting unit 207A.

[0104] Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, where the blocks may represent (1) stages of a process in which operations are performed or (2) sections of an apparatus responsible for performing the operations. Particular stages and sections may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable medium, and / or a processor provided with computer-readable instructions stored on a computer-readable medium. Dedicated circuitry may include digital and / or analog hardware circuitry, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuitry may include reconfigurable hardware circuitry, including logical AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, memory elements such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0105] A computer-readable medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that the computer-readable medium having instructions stored thereon comprises an article of manufacture containing instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable media may include electronic, magnetic, optical, electromagnetic, and semiconductor storage media. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, and the like.

[0106] The computer readable instructions may include either assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0107] The computer-readable instructions may be provided to a processor or programmable circuit of a programmable data processing device, such as a computer, locally or over a wide area network (WAN) such as a local area network (LAN) or the Internet, and the computer-readable instructions may be executed to create means for performing the operations specified in the flowcharts or block diagrams. Here, the computer may be a personal computer (PC), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, a special-purpose computer, or the like, or may be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system, and is a broad definition of computer. In a distributed computing system, the multiple computers collectively execute a program by each executing a portion of the program and passing data between the computers as needed during program execution.

[0108] Examples of processors include a computer processor, a central processing unit (CPU), a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, etc. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute a program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0109] 9 illustrates an example of a computer 1200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 1200 may cause the computer 1200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0110] A computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, a graphics controller 1216, and a display device 1218, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224 such as a hard disk drive, a DVD-ROM drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The computer also includes legacy input / output units such as a ROM 1230 and a keyboard 1242, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0111] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller 1216 itself, and causes the image data to be displayed on the display device 1218.

[0112] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD-ROM drive 1226 reads programs or data from a DVD-ROM 1227 and provides the programs or data to the storage device 1224 via the RAM 1214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0113] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0114] The programs are provided by a computer-readable medium such as a DVD-ROM 1227 or an IC card. The programs are read from the computer-readable medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or a method may be configured by implementing information manipulation or processing in accordance with the use of the computer 1200.

[0115] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer processing area provided in the RAM 1214, the storage device 1224, the DVD-ROM 1227, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0116] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, the DVD-ROM drive 1226 (DVD-ROM 1227), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data in the RAM 1214. The CPU 1212 then writes back the processed data to the external recording medium.

[0117] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. CPU 1212 may perform various types of processing on data read from RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to RAM 1214. CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, CPU 1212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0118] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 1200 via the network.

[0119] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0120] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0121] 1,1A system 2,2A operation support device 3 Display device 4 Equipment 40 equipment 201,201A Operation acquisition part 202 Equipment control section 203,203A Display control unit 204 Storage section 205 Proficiency Acquisition Department 206 Role Setting Department 207A Limit setting section 208A Operation Restriction Unit 209A Release part 1200 Computer 1210 host controller 1212 CPU 1214 RAM 1216 Graphics Controller 1218 Display Devices 1220 Input / Output Controller 1222 communication interface 1224 Storage device 1226 DVD-ROM drive 1227 DVD-ROM 1230 ROM 1240 Input / Output Chip 1242 keyboard 2031, 2031A Virtual space display control unit 2032, 2032A Avatar display control unit A1 Veteran Avatar A2 Newcomer Avatar P1, P2 pipes B1, B2 valves S flow meter

Claims

1. a first display control unit that displays a common virtual space for each of a plurality of operators; an operation acquisition unit that acquires an operation for an object in the virtual space from one operator; a second display control unit that displays an avatar representing the operator in the virtual space for each operator and causes the avatar to perform an action corresponding to the operation; An apparatus comprising:

2. the second display control unit displays a plurality of avatars representing the plurality of operators in the virtual space; The device according to claim 1 , wherein the first display control unit displays, for each operator, a view of the virtual space as seen from an avatar corresponding to that operator.

3. The device according to claim 2, wherein the first display control unit, in response to an operation from an operator, causes the view of the virtual space from the one avatar to be displayed to each of the plurality of operators other than the one operator.

4. The device according to claim 2, wherein the first display control unit, in response to an operation from an operator, directs the line of sight of each of the other avatars among the plurality of avatars that is different from the one avatar toward an area of ​​the virtual space that is included in the field of view of the one avatar as seen from the virtual space.

5. The device according to claim 2, wherein the first display control unit causes each of the plurality of operators, other than the one operator, to display a mark indicating an area of ​​the virtual space that is included in the field of view of the one avatar when looking at the virtual space.

6. The device according to claim 5, wherein the first display control unit distinguishably displays an area of ​​the virtual space that is included in the field of view of the one avatar when looking at the virtual space within the virtual space displayed to each of the other operators other than the one operator among the plurality of operators.

7. the operation acquisition unit acquires an operation on an object in the virtual space from each operator; The device according to claim 2 , wherein the second display control unit causes an avatar corresponding to an operator who performed an operation acquired by the operation acquisition unit to perform an action corresponding to the operation.

8. The device according to claim 7 , further comprising an action limiting unit that limits the actions of other avatars, different from the one avatar, among the avatars of the plurality of operators, to content that is preset for the other avatars.

9. a proficiency level acquisition unit that acquires proficiency levels of other operators, different from the one operator, in operating an object, among the plurality of operators; a first setting unit that sets the content of the action that the other avatar can perform to the content of the action that is associated in advance with the proficiency level acquired for the other operator corresponding to the other avatar; The apparatus of claim 8 further comprising:

10. The device according to claim 8 , further comprising a release unit that releases the restriction on the operation imposed by the operation limiting unit in response to an operation by the one operator.

11. The device according to claim 7 , wherein the operation acquisition unit includes a cancellation unit that cancels an operation on an object in the virtual space from another operator among the plurality of operators that is different from the one operator, in response to an operation from the one operator.

12. a skill level acquisition unit that acquires a skill level of each operator in operating an object; a second setting unit that sets the operator with the highest skill level among the plurality of operators as the first operator; The apparatus of claim 1 further comprising:

13. the virtual space is a three-dimensional model of a facility having at least one device as an object; The device comprises: The device according to claim 1 , further comprising: a device control unit that controls a device corresponding to an object in the virtual space in response to an operation on the object acquired by the operation acquisition unit.

14. The device according to claim 1 , further comprising a storage unit that stores the content of each operation acquired by the operation acquisition unit.

15. a first display control step of displaying a common virtual space for each of a plurality of operators; an operation acquisition step of acquiring an operation for an object in the virtual space from one operator; a second display control step of displaying an avatar representing the operator in the virtual space for each operator and causing the avatar to perform an action corresponding to the operation; A method for providing the above.

16. When executed by a computer, the computer a first display control unit that displays a common virtual space for each of a plurality of operators; an operation acquisition unit that acquires an operation for an object in the virtual space from one operator; a second display control unit that displays an avatar representing the operator in the virtual space for each operator and causes the avatar to perform an action corresponding to the operation; A program that functions as a

Images