Simulator system

Abstract

A simulator system includes: a computer that reproduces a construction machine having an accessory device and an environment surrounding the construction machine in a virtual space; a display unit that displays the construction machine and the surroundings; and an input device that receives an operation by an operator. The computer includes: an object processing unit that moves an object of at least one of the construction machine and the surrounding environment so that the construction machine is in an unsafe state; and an operation determination unit that determines the suitability of the operation input by the input device on the basis of the content of the operation input by the operator to the input device when the construction machine is in the attention-requiring state before the construction machine has changed to the unsafe state.

Description

TECHNICAL FIELD

[0001] The present application relates to a simulator system for simulating an operation of a construction machine reproduced in a virtual space. BACKGROUND

[0002] For example, in Patent Literature 1, a driving teaching result evaluation system that evaluates a driving operation content in a driving simulator is described.

[0003] For example, it is desirable in a simulation of a construction machine to evaluate whether an operation performed by an operator is an appropriate operation corresponding to a situation of at least one of the construction machine and a surrounding environment.

[0004] PRIOR ART DOCUMENTS PATENT LITERATURE Patent Literature 1: Japanese Patent Publication No. 3761484 SUMMARY

[0005] An object of the present application is to provide a simulator system capable of evaluating an appropriateness of an operation by an operator.

[0006] The simulator system according to the present application includes a computer that reproduces a construction machine having an attached device and a surrounding environment of the construction machine in a virtual space, a display section that displays the construction machine and the surrounding environment, and an input device that receives an operation by an operator for causing the construction machine to work in the virtual space. The computer includes an object processing section that is capable of moving an object of at least one of the construction machine and the surrounding environment to cause the construction machine to become an unsafe state, and an operation determining section that determines an appropriateness of an operation input to the input device based on a content of the operation input to the input device by the operator at a caution state before the construction machine becomes the unsafe state. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 is a block diagram showing a simulator system 1.

[0008] Figure 2 is a diagram showing a construction machine 4 and a target object displayed by an operator device 10 shown in Figure 1

[0009] Figure 3 is a diagram showing a specific example of the operator device 10 shown in Figure 1

[0010] Figure 4 is a diagram showing an operation device 11a of the operator device 10 shown in Figure 3

[0011] is a diagram showing an operation device 11a of the operator device 10 shown in​​Figure 5 is a diagram indicating that a person approaches Figure 2 a graph showing the state of the construction machine 4.

[0012] Figure 6 is a diagram indicating Figure 3 a diagram showing a menu screen displayed by the operator device 10.

[0013] Figure 7 is a diagram indicating a display example of a display unit in a field work mode selected from Figure 6 a menu screen.

[0014] Figure 8 is a diagram indicating Figure 1 a flowchart of a menu process of the computer 13.

[0015] Figure 9 is a flowchart of a driving teaching process executed in Figure 8 the menu process.

[0016] Figure 10 is a flowchart of a safety training process executed in Figure 8 the menu process.

[0017] Figure 11 is a flowchart of an operation determination process executed in Figure 10 the safety training process. DETAILED DESCRIPTION

[0018] An embodiment of the simulator system 1 according to the present application will be described with reference to the drawings.

[0019] As shown in Figure 1 , the simulator system 1 is a system for simulating ground operation of a construction machine 4 reproduced in a virtual space. First, the construction machine 4 as an operation object of the simulator system 1 will be described.

[0020] As shown in Figure 2As shown, the construction machine 4 is a three-dimensional model that simulates a real construction machine. The construction machine 4 is a construction machine that performs a construction work, for example, and can be an excavator, can be a crane, can be a bulldozer, can be a loader, and can be a dump truck (transport vehicle). Hereinafter, a case where the construction machine 4 is an excavator is mainly described. The construction machine 4 can also operate based on information that is set in advance by the computer 13. The construction machine 4 can also operate according to manual operation by a user. The construction machine 4 can also travel, the upper swing body 41b (described later) can also swing relative to the lower traveling body 41a (described later), and the posture of the attachment 43 can also change. The construction machine 4 can also spray a washing liquid (not shown) to the outer surface of the windshield. The construction machine 4 can also wipe rainwater and the washing liquid (not shown) and the like that adhere to the outer surface of the windshield. The construction machine 4 can also have a lighted vehicle light (for example, a headlamp that can illuminate the front of the construction machine 4) (not shown). Furthermore, the construction machine 4 can also not operate (it can also be simply arranged in the virtual space).

[0021] The construction machine 4 includes a machine main body 41 and an attachment 43.

[0022] The machine main body 41 is a main body portion of the construction machine 4. The machine main body 41 includes a lower traveling body 41a and an upper swing body 41b.

[0023] The lower traveling body 41a is capable of traveling on a traveling surface (ground or the like). The lower traveling body 41a can be provided with a track 41a1, and can be provided with a wheel (not shown).

[0024] The upper slewing body 41b is rotatable relative to the lower traveling body 41a. The upper slewing body 41b may also include a cab 41b2. The cab 41b2 is the part where the operator of the construction machinery sits. The cab 41b2 may also be fixed to the bottom (slewing frame) of the upper slewing body 41b. The cab 41b2 may also be movable relative to the bottom of the upper slewing body 41b. In this case, the cab 41b2 can move horizontally up and down relative to the bottom of the upper slewing body 41b, can rotate horizontally up and down, or can move horizontally up and down and rotate. For example, the cab 41b2 may be a lift-type cab, a linkage-type cab, or a tilt-type cab. As a lift-type cab, the cab 41b2 can move horizontally up and down relative to the bottom of the upper slewing body 41b. As a linkage-type cab, the cab 41b2 is mounted at the bottom of the upper slewing body 41b via a linkage mechanism. As a linkage-type cockpit, the cockpit 41b2 can move vertically and horizontally relative to the bottom of the upper rotating body 41b due to the operation of the linkage mechanism. As a tilting-type cockpit, the cockpit 41b2 can rotate vertically (tilt) relative to the bottom of the upper rotating body 41b about a rotation axis extending laterally. The cockpit 41b2 may also have accessories installed on the main body of the cockpit 41b2. These accessories may be, for example, protective parts for protecting the upper surface or front window of the cockpit 41b2 from flying objects.

[0025] The auxiliary device 43 is the working mechanism (working device). The auxiliary device 43 is capable of undulating relative to the upper rotating body 41b (and can rotate in the up and down direction). The auxiliary device 43 includes the boom 43a1, the stick 43a2, and the distal auxiliary device 43c.

[0026] The boom 43a1 is rotatably mounted on the upper slewing body 41b. The stick 43a2 is rotatably mounted on the boom 43a1.

[0027] A remote attachment 43c is disposed at the distal end of the attachment 43. The remote attachment 43c is rotatably mounted to the boom 43a2. The remote attachment 43c may also be a bucket used for digging or excavating objects. The remote attachment 43c may be a device for clamping the work object (grab bucket, crusher, etc.), a device for crushing the work object (breaker hammer, etc.), or a lifting magnet for attracting metal work objects. The object that is the target of the operation performed by the remote attachment 43c (work object) may be sand, stone, wood, metal, resin, waste, or a structure (concrete block, etc.).

[0028] Simulator system 1 is a system for simulating the operation of construction machinery 4 in a virtual space. Specifically, simulator system 1 can be used to teach operators how to operate construction machinery 4. Additionally, simulator system 1 can be used to provide safety training to operators on the construction machinery 4. In other words, simulator system 1 has both driving instruction functions (instructing operators on how to operate construction machinery 4) and safety training functions (providing operators with safety education on the construction machinery 4). The "operator" is the user of the simulator system. Simulator system 1 may also include at least one of a personal computer, tablet computer, or smartphone.

[0029] like Figure 1 As shown, the simulator system 1 includes an operator device 10 and a computer 13. Additionally, the simulator system 1 may also include a server device 20.

[0030] Operator device 10 is a device operated by an operator. Operator device 10 can be operated when the operator receives driving instruction for construction machinery 4. Additionally, operator device 10 can be operated when the operator receives safety training for construction machinery 4. Operator device 10 can also be used to remotely operate the actual machinery corresponding to construction machinery 4, i.e., the actual construction machinery 40.

[0031] The operator device 10 includes an input device 11 and an output device 12.

[0032] Input device 11 is a device for inputting information into computer 13. Input device 11 is operated by an operator. Input device 11 can also be used for starting / ending driving lessons or safety training. Input device 11 can also be used for repeatedly performing the same driving lessons or safety training. Input device 11 may also include at least one of a mouse, keyboard, touch screen, joystick, pedal, and game controller. Input device 11 may also be a touch screen provided on the display unit 12a (e.g., a monitor) described later. Input device 11 can also be used for remote operation of construction machinery 40. Input device 11 can also receive instructions to start and end work on construction machinery 4.

[0033] The input device 11 includes an operating device 11a, a locking device 11b, a horn operating unit 11c, and an engine speed setting device 11d.

[0034] The operating device 11a is a device that receives operations from the operator to make the engineering machinery 4 work in the virtual space. The operating device 11a includes a travel lever 11a2 and an auxiliary device operating lever 11a1.

[0035] The travel lever 11a2 is operated to move the construction machinery 4. For example... Figure 3As shown, the walking bar 11a2 is positioned in front of the motion seat 12b2 (described later). The walking bar 11a2 has a pair of bars arranged side-by-side laterally. A walking pedal (see reference) is fixed to the lower part of the walking bar 11a2. Figure 4 ).

[0036] The auxiliary device operating lever 11a1 is operated to control the movement of the auxiliary device 43. The auxiliary device operating lever 11a1 includes a left operating lever 11a1A and a right operating lever 11a1B. The left operating lever 11a1A is located in front of the left frame of the motion seat 12b2. The right operating lever 11a1B is located in front of the right frame of the motion seat 12b2. The left operating lever 11a1A and the right operating lever 11a1B can perform multiple operations such as forward / backward and left / right movements. (This is used for operating engineering machinery 4 (refer to...)) Figure 2 The functions of rotating the upper rotating body 41b, operating the boom 43a1, operating the stick 43a2, and operating the remote auxiliary device 43c are assigned to any one of the aforementioned operations. The assignment of operations can also be changed.

[0037] The locking device 11b is activated to lock the operation of the construction machinery 4. The locking device 11b is located in front of the left operating lever 11a1A. The locking device 11b is, for example, rod-shaped. The locking device 11b can also lock when the lever is raised (opened), so that the construction machinery 4 will not move even if the operating levers are operated. The locking device 11b can also be released when the lever is lowered (closed).

[0038] The horn operating unit 11c is operated to sound the horn. Figure 3 In the example shown, the horn operation unit 11c is located on the left operating lever 11a1A. The horn operation unit 11c is, for example, a button. Engine speed setting device 11d (see reference) Figure 1 ) is a device used to set the engine speed of engineering machinery 4 in virtual space (in Figure 3 (Not shown in the diagram). The engine speed setting device 11d changes the travel speed (working speed) of the construction machinery 4.

[0039] like Figure 1 As shown, the output device 12 outputs according to the instructions input from the computer 13. The output device 12 includes a display unit 12a and a notification unit 12b.

[0040] like Figure 3 As shown, display unit 12a is a device for displaying information to the operator. Display unit 12a is, for example, a monitor (screen). Display unit 12a displays computer 13 (see reference). Figure 1The display unit 12a displays the virtual space calculated by the computer 13, etc. The display unit 12a is used to display the virtual space reproducing the construction machinery 4. The display unit 12a can also be used as a guide for driving instruction and safety training. The display unit 12a can also be used to explain the operation method of the construction machinery 4. The following "display" refers to the display performed by the display unit 12a at the instruction of the computer 13.

[0041] The display unit 12a includes a display unit 12a1. The display unit 12a1 is located in front of the motion seat 12b2. The display unit 12a1 includes multiple (seven in this embodiment) displays. Specifically, the display unit 12a1 includes a central main display 12a1A, a left main display 12a1B, a right main display 12a1C, an upper display 12a1D divided into three parts horizontally, and a lower display 12a1E. Some or all of the displays in the display unit 12a1 may also be touchscreens.

[0042] Notification Department 12b (see reference) Figure 1 The notification unit 12b is a device used to notify the operator. The output from the notification unit 12b may be, for example, one or more of a sound output and a vibration output. The notification unit 12b may, for example, provide feedback on the operator's actions. The notification unit 12b may, for example, report the actions the operator should input. The output from the notification unit 12b may also be displayed on the display unit 12a.

[0043] The notification unit 12b includes a speaker 12b1 and a motion seat 12b2.

[0044] The motion seat 12b2 operates to provide feedback to the operator. The operation of the motion seat 12b2 for feedback involves one or both of changes in the tilt angle of the seating area and vibration of the seating area. When the construction machinery 4 moves within the virtual space, the motion seat 12b2 provides feedback to the operator by reproducing the movement corresponding to the movement. The motion seat 12b2 is controlled, for example, to achieve the same tilt angle as the construction machinery 4 by receiving a signal representing the tilt angle of the construction machinery 4. Furthermore, the motion seat 12b2 transmits feedback to the operator of the operation input device 11 through vibration. That is, the motion seat 12b2 provides feedback to the operator on the operating status of the construction machinery 4 by vibrating according to the operator's operation of the input device 11. The motion seat 12b2 is in the form of a high-backed chair with armrests. The motion seat 12b2 can also be, for example, in any form that allows the operator to sit comfortably, such as a low-backed chair without a headrest or a backless chair.

[0045] Speaker 12b1 outputs sound. Speaker 12b1 is used, for example, to output voice instructions and horn sounds. Figure 3In the example shown, the speaker 12b1 is located on the left side of the left frame of the motion seat 12b2.

[0046] Computer 13 (Reference) Figure 1 The computer 13 is a computer that includes a processor for inputting and outputting signals, performing calculations, and storing information. The computer 13 recreates the construction machinery 4 and its surrounding environment in a virtual space. The arithmetic unit 13b executes the program stored in the storage unit 13a, thereby realizing the various functions of the computer 13. These functions include, for example, the functions of the object processing unit 13b1, target determination unit 13b2, target prompting unit 13b3, output control unit 13b4, operation determination unit 13b5, and remote operation unit 13b6, which will be described later. The computer 13 can be connected to other devices (e.g., input device 11, output device 12) via wireless communication or wired communication. Information is input from the input device 11 to the computer 13. The computer 13 outputs information to the output device 12. The computer 13 can also exchange information with the server device 20. This exchange between the computer 13 and the server device 20 can be conducted via, for example, mobile networks, optical networks, wireless LANs (Local Area Networks), or wired LANs.

[0047] like Figure 2 As shown, computer 13 recreates construction machinery 4 in virtual space. Construction machinery 4 can be a complete recreation of the actual construction machinery 40, or a recreation of a portion of the actual construction machinery 40. Additionally, computer 13 recreates the surrounding environment of the actual construction machinery 40 in virtual space. The recreated surrounding environment may include, for example, people, ground, work objects, obstacles, transport vehicles, walls, and hilltops. Hereinafter, the elements of the construction machinery 4 and its surrounding environment recreated in virtual space will sometimes be collectively referred to as objects. An object is a model obtained by modeling the elements of the construction machinery 4 and its surrounding environment; it can be a three-dimensional model or a two-dimensional model.

[0048] For example, elements such as the boom 43a1, stick 43a2, and distal attachment 43c of the auxiliary device 43 are all objects of the construction machinery 4. Furthermore, elements such as people or the ground recreated around the construction machinery 4 are all objects of the surrounding environment. For example, in Figure 5 In the example shown, a human object 30P, recreating a person, is generated as an object in the surrounding environment. The computer 13 has object processing functions for controlling such objects.

[0049] The computer 13 includes a storage unit 13a and an arithmetic unit 13b.

[0050] Storage Unit 13a (refer to) Figure 1The storage unit 13a stores various information. It stores the program of the simulator system 1. The storage unit 13a pre-stores appropriate operations corresponding to one or both of the conditions of the construction machinery 4 and its surrounding environment. The storage unit 13a pre-stores notifications corresponding to the appropriateness of the operations handled by the input device 11 as determined by the operation determination unit 13b5. The storage unit 13a stores the operation target of the construction machinery 4. The operation target is the destination of the object's movement when an appropriate operation is performed, i.e., when the desired operation corresponding to the pre-stored appropriate operation is input to the input device 11; it can be set to different locations depending on the situation. That is, the storage unit 13a stores setting patterns for multiple operation targets used differently depending on the situation. The storage unit 13a stores reports. Reports can also be stored in the storage unit 13a of the server device 20. The storage unit 13a stores authentication information for login. Authentication information can also be stored in the storage unit 13a of the server device 20.

[0051] Arithmetic Unit 13b (refer to) Figure 1 The calculation unit 13b performs various calculations and judgments. The calculation unit 13b includes an object processing unit 13b1, a target determination unit 13b2, a target prompting unit 13b3, an output control unit 13b4, an operation determination unit 13b5, and a remote operation unit 13b6. The object processing unit 13b1 controls the object. For example, when an object (hereinafter referred to as a movable object) moves in response to an operator's operation on the input device 11, the object processing unit 13b1 controls the movement of the movable object. The target determination unit 13b2 determines whether the movable object has moved to its operation target. Specifically, the target determination unit 13b2 determines whether the movable object has reached the same position as the target object when the movable object has moved to its operation target. Based on the current state of the construction machinery 4, the target prompting unit 13b3 sequentially selects a setting style from multiple setting styles related to the target object stored in the storage unit 13a, and prompts for the target object corresponding to the selected setting style. The output control unit 13b4 determines the content to be output to the output device 12. The operation determination unit 13b5 determines the appropriateness of the operation received by the input device 11. The remote operation unit 13b6 controls the remote operation of the engineering machinery 40 via the input device 11. The calculation unit 13b may also include an authentication processing unit (not shown). The authentication processing unit processes the authentication of user login. The authentication processing unit uses user identification information to identify the user. The user identification information may also be stored in the storage unit 13a in correspondence with the report. The user identification information may be, for example, a username and user ID. The authentication process may also be executed in the calculation unit 13b of the server device 20.

[0052] Server device 20 (reference) Figure 1The client device 19 is a computer that operates according to instructions from the client device 19. The client device 19 consists of the aforementioned operator device 10 (input device 11 and output device 12) and the computer 13. The server device 20 may also store authentication information for the arithmetic unit 13b to process user authentication. In addition, the server device 20 may also perform authentication processing for authenticated users. The functions of the simulator system 1, such as the object processing unit 13b1, target determination unit 13b2, target prompting unit 13b3, output control unit 13b4, operation determination unit 13b5, and remote operation unit 13b6, may also be implemented by a combination of the client device 19 and the server device 20. At least one of the storage unit 13a and the arithmetic unit 13b of the computer 13 may also be provided in the client device 19 and the server device 20. Multiple server devices 20 may also be provided. Furthermore, the simulator system 1 may not have a server device 20. The input device 11 may also be provided in the server device 20 (not shown). In this case, various information can also be input from the input device 11 set on the server device 20 to the computer 13 set on the client device 19.

[0053] 40 actual construction machinery units (for reference) Figure 1 The actual machine 40 is constructed in the same manner as the construction machinery 4 simulated by the simulator system 1. The simulator system 1 may also be equipped with a remote operation device for remotely operating the construction machinery 40. The simulator system 1 may also be installed on the remote operation device. The remote operation device includes an input device 11, a computer 13, and an output device 12. The input device 11 has, for example, an operation device 11a including an operating joystick or an operating pedal, and outputs operation signals (e.g., joystick operation signals or pedal operation signals) corresponding to the operation of the operation device 11a. During remote operation, the computer 13 exchanges signals with the actual machine that is the object of remote operation via a communication unit. During remote operation, the computer 13 sends signals (work command signals) to the actual machine to operate the working device (corresponding to the auxiliary device 43) or the traveling device (corresponding to the lower traveling body 41a) of the construction machinery 40 based on the operation signals input from the input device 11. During remote operation, the computer 13 receives image data from the camera mounted on the construction machinery 40 or signals output by the sensors mounted on the construction machinery 40 via the communication unit. When operating remotely, the output device 12 (specifically the display unit 12a1) outputs image data (data acquired by the computer 13) from the camera mounted on the engineering machinery 40.

[0054] The simulator system 1 also includes a remote control device, allowing users to operate the construction machinery 4 in a virtual space using the same device used for remotely operating the actual construction machinery 40. This allows users to experience the feel of operating the actual construction machinery 40 by operating the virtual machinery. As a result, it promotes user proficiency in operating the remote control device. Furthermore, it makes it easier to predict what might happen when operating the actual construction machinery 40 at the work site using the remote control device.

[0055] The simulator system 1 (primarily computer 13) performs the following processing. The simulation program instructs computer 13 to execute the following processing. The simulator system 1 implements the method for performing the following processing.

[0056] (Object processing function) The computer 13 (specifically, the object processing unit 13b1) has object processing functions for processing objects. Furthermore, each function of the simulator system 1 is equivalent to a step in the simulation program and simulation method. For example, the object processing function for processing objects is equivalent to the object processing steps. The object processing function and object processing steps for object 30 include the processing of displaying the object on the display unit 12a1.

[0057] Computer 13 causes at least one of the construction machinery 4 and its surrounding environment to move. Construction machinery 4 can operate according to operator commands or move automatically. For example, construction machinery 4 can also operate using machine control systems (MC). Specifically, a work plan is set for computer 13. Then, the operator, for example, operates only the auxiliary device 43 (see reference 13). Figure 2 The computer 13 automatically controls elements not operated by the operator (e.g., boom 43a1 only). In this case, the computer 13 automatically controls elements not operated by the operator (e.g., stick 43a2, remote attachment 43c) to cause the construction machinery 4 to perform operations according to the work plan. At this time, the computer 13 controls the operation of the construction machinery 4 based on its posture (the same applies in the case of automatic driving). As a result, the construction machinery 4 performs operations according to the work plan. Alternatively, for example, the construction machinery 4 can also operate using automatic driving. In this case, the computer 13 controls the operation of the construction machinery 4 to cause it to automatically perform operations according to the work plan.

[0058] (The movement of the object corresponding to the operation) Computer 13 according to operating device 11a (refer to) Figure 3 The computer 13 can make the object work based on the input received by the walking stick 11a2 (see reference). Figure 3 The operation of ) causes the lower walking body 41a (refer to) Figure 2The computer 13 can also move the construction machinery 4 by operating the left control lever 11a1A (see reference). Figure 3 The left and right directions of the operation cause the upper rotating body 41b (refer to) to rotate. Figure 2 The computer 13 can also rotate the boom 43a2 by operating the left control lever 11a1A in the forward or backward direction. The computer 13 can also rotate the boom 43a2 by operating the right control lever 11a1B (see reference). Figure 3 The forward and backward movement of the boom 43a1 (refer to) causes the boom to move forward and backward. Figure 2 The computer 13 can also rotate, for example, by operating the right-hand control lever 11a1B in the left and right directions, causing the remote accessory device 43c (see reference) to rotate. Figure 2 Rotate.

[0059] Computer 13 changes the operating speed of the object operated by input device 11 based on the input amount. For example, computer 13 changes the operating speed of auxiliary device 43 based on the input amount of auxiliary device joystick 11a1. Computer 13 changes the operating speed of construction machinery 4 based on the engine speed set by engine speed setting device 11d. For example, computer 13 changes the operating speed of the basic auxiliary device 43 and the travel speed of construction machinery 4 based on engine speed.

[0060] (Display content of the display section) like Figure 4 As shown, the object of the construction machinery 4 operated via the input device 11 is displayed on the display unit 12a (specifically, the display unit 12a1). The object of the construction machinery 4 is displayed in a form seen from a preset viewpoint. The preset viewpoint may also be the viewpoint of the operator sitting in the driver's seat inside the cab 41b2. In this case, the displayed construction machinery 4 may also be a part of the construction machinery 4. For example, the interior components of the cab 41b2 that are visible from the driver's seat, or the auxiliary devices 43 that are visible through the windows of the cab 41b2, may also be displayed.

[0061] The viewpoint can be set from above (top view), from the side (side view), or from an obliquely upward view (stereoscopic view). Multiple viewpoints can also be displayed for the engineering machinery 4. For example, more than one of the stereoscopic view, side view, and top view can be displayed.

[0062] In addition to displaying the object of the construction machinery 4, the display unit 12a also displays a target object representing the appropriate destination for the movable object, which is a movable object. As described above, the target object is the object representing the destination for the movable object when the operator performs the desired operation on the input device 11. Furthermore, objects representing the surrounding environment of the construction machinery 4 are also displayed on the display unit 12a. These surrounding environment objects include, for example, people, the ground, the work object, obstacles, transport vehicles, walls, and hilltops.

[0063] The accessory device 43 displayed on the display unit 12a includes an accessory device movable body 30b, such as a movable part equivalent to a stick 43a2 or a remote accessory device 43c. The accessory device movable body 30b is one of the movable bodies that moves in response to operation of the input device 11.

[0064] like Figure 4 As shown, the display unit 12a also displays objects simulating the various components of the input device 11. These objects of the input device 11 are generated within the cab 41b2 of the construction machinery 4. The objects of the input device 11 include an input device movable body 30a that simulates the movable parts of the operating device 11a, such as the auxiliary device operating lever 11a1 and the travel lever 11a2. The input device movable body 30a is one of the movable bodies that moves in response to the operation of the input device 11. The input device movable body 30a includes a travel lever object 30a2 simulating the travel lever 11a2 and an auxiliary device operating lever object 30a1 simulating the auxiliary device operating lever 11a1. The travel lever object 30a2 is displayed in a manner that moves synchronously with the operation of the travel lever 11a2. The auxiliary device operating lever object 30a1 is displayed in a manner that moves synchronously with the operation of the auxiliary device operating lever 11a1. The movable part of the auxiliary device 43, namely the movable body 30b of the auxiliary device, is shown to move synchronously with the operation when the auxiliary device operating lever 11a1 is operated.

[0065] Although not shown, the computer 13 can also instruct the display unit 12a to display information that indicates the current position of the movable part, such as the joystick 11a1 or the travel joystick 11a2 of the input device 11, in relation to the range of motion of that movable part. For example, the computer 13 can display a defined graphic object representing the current position of the movable part of the input device 11 on an object that simulates the range of motion of that movable part, thereby indicating the current position in relation to the range of motion.

[0066] The computer 13 may also cause the display unit 12a to display information that indicates the current position of the accessory 43 in relation to its movable range. For example, the computer 13 may display a defined graphic object representing the current position of the accessory 43 on an object that simulates the range of movable range of the accessory 43, thereby indicating the current position in relation to the movable range.

[0067] The computer 13 may also highlight the movable object that is the subject of the operation received by the input device 11. The computer 13 may also adjust the viewing angle of the display unit 12a1 so that the remote auxiliary device 43c performing the operation is always displayed.

[0068] The display content of each display of the display unit 12a1 can be configured in various ways. For example, such as Figure 4 As shown, the central main display 12a1A displays the view from the driver's seat of the construction machinery 4. For example, the lower display 12a1E displays a side view of the construction machinery 4. For example, a guide to driving instruction and safety training may also be displayed on any of the displays in the display unit 12a1. For example, a perspective view of the construction machinery 4 may also be displayed on any of the displays in the display unit 12a1. For example, operating instructions for the construction machinery 4 or a top view of the construction machinery 4 may also be displayed on any of the displays in the display unit 12a1.

[0069] (Driving instruction function) Computer 13 indicates the operational target of the movable body of the engineering machinery 4 and determines whether the movable body has reached the operational target. For example, computer 13 (specifically target determination unit 13b2) determines whether the movable body has reached the same position as the target object representing the operational target.

[0070] The operation target may also include the relative angle of the movable body that is the object of operation (e.g., the angle of the stick 43a2 relative to the boom 43a1, etc.). The operation target may also include the angular velocity of the rotation or slewing of the movable body. The operation target may also include the coordinates to which the movable body should point. The operation target may also be displayed as a target object. The target object may also be displayed through images from multiple viewpoints.

[0071] The target object is the appropriate destination for the movable body of the engineering machinery 4, i.e., the object to be operated. The operator operates the input device 11 with the target object as the target, causing the engineering machinery 4 to work in the virtual space. The target object is displayed superimposed on the movable body. The target object has the same shape as the movable body. The target object may also have the same or substantially the same appearance as the movable body. The aforementioned "same shape" includes, for example, a shape that converts the movable body to a semi-transparent state, and a shape that only displays the outline of the movable body.

[0072] The target object is, for example, a target object 31b with an accessory device and a target object 31a with an input device.

[0073] like Figure 2 As shown, the auxiliary device target object 31b is an object representing the operational target of the auxiliary device 43, that is, an object representing the destination of the movable body 30b of the auxiliary device when an operation (desired operation) consistent with the pre-stored appropriate operation is performed on the input device 11. The movable body 30b of the auxiliary device is an object of the movable part of the auxiliary device 43, such as the boom 43a1, the stick 43a2, and the distal auxiliary device 43c, and is a movable body of the construction machinery 4. The auxiliary device target object 31b can be displayed on the display unit 12a1 (see reference) in a state of overlapping with the movable body 30b of the auxiliary device. Figure 2 ).like Figure 4 As shown, depending on the viewpoint, the target object 31b of the auxiliary device may not overlap with the display of the movable object 30b of the auxiliary device.

[0074] like Figure 4 As shown, the input device target object 31a is an object representing the operation target of the input device 11, that is, an object representing the destination of the movable body 30a of the input device when an operation (desired operation) consistent with a pre-stored appropriate operation is performed on the input device 11. The movable body 30a of the input device is an object of the movable part of the input device 11, such as the auxiliary device operating lever 11a1 or the travel lever 11a2, and is a type of movable body of the engineering machinery 4. That is, the movable body 30a of the input device includes an auxiliary device operating lever object 30a1 that simulates the auxiliary device operating lever 11a1 and a travel lever object 30a2 that simulates the travel lever 11a2. The input device target object 31a includes an auxiliary device operating lever target object 31a1 that represents the operation target of the auxiliary device operating lever 11a1 and a travel lever target object 31a2 that represents the operation target of the travel lever 11a2. The input device target object 31a can be displayed on the display unit 12a1 in a state of overlapping with the movable body 30a of the input device. Figure 4 In the example shown, the auxiliary device operating lever target object 31a1 is displayed in a state overlapping with the auxiliary device operating lever object 30a1. Similarly, the travel lever target object 31a2 is displayed in a state overlapping with the travel lever object 30a2.

[0075] Computer 13 (specifically, target prompting unit 13b3) selects and prompts for a target object (operation target). Computer 13 can also prompt for the target object in various sequences. Specific examples are shown below. For instance, based on the current state of the construction machinery 4, computer 13 can sequentially select a setting style from multiple pre-stored setting styles related to the target object and prompt for the target object corresponding to the selected setting style. Computer 13 can also cause target objects corresponding to multiple setting styles to be displayed sequentially. Computer 13 can also end the prompting for the target object by target prompting unit 13b3 based on specific input (e.g., input to the touchscreen). Computer 13 can also re-display the target object corresponding to the previously selected setting style based on specific input (e.g., input to the touchscreen).

[0076] While observing the construction machinery 4 and the target object displayed on the display unit 12a, the operator operates the input device 11 to bring the movable part of the construction machinery 4 closer to the target object. The computer 13 may also only accept operations corresponding to the movable part (operation object) of the target object.

[0077] Computer 13 (specifically, target determination unit 13b2) determines whether the movable body has reached the same position as the target object. The movable body and the target object may not be perfectly aligned. That is, the movable body may be considered to have reached the target object when it approaches within a specified range. Determining whether the movable body has reached the target object is equivalent to determining whether the movable body (operated object) has been moved to the position of the operating target. Computer 13 determines whether the auxiliary device movable body 30b has reached the auxiliary device target object 31b. Computer 13 determines whether the input device movable body 30a has reached the input device target object 31a. For example, computer 13 determines whether the walking stick object 30a2 has reached the walking stick target object 31a2. For example, computer 13 determines whether the auxiliary device operating stick object 30a1 has reached the auxiliary device operating stick target object 31a1. For example, computer 13 determines whether the input device movable body 30a has reached the input device target object 31a before the auxiliary device movable body 30b reaches the auxiliary device target object 31b.

[0078] The computer 13 can also provide feedback on the judgment result to the operator. Feedback to the operator includes reporting that the input device 11 has been operated correctly. For example, it can also provide information indicating that the movable object has reached the same position as the target object. Feedback on the operator's operation includes reporting that it has not been operated correctly. For example, if no operation is input to the input device 11 within a specified time after the target object is indicated, the computer 13 can also issue a warning. For example, if an operation that cannot make the movable object reach the same position as the target object is input to the input device 11 after the target object is indicated, the computer 13 can also issue a warning. An operation that cannot make the movable object reach the same position as the target object is, for example, an operation on an incorrect target object, such as mistakenly operating the walking lever 11a2 when the auxiliary device operating lever 11a1 should be operated. For example, if the movable object does not reach the same position as the target object within a specified time after the target is indicated, the computer 13 can also issue a warning. The computer 13 can provide feedback by outputting voice guidance, by outputting sound such as a warning tone, or by displaying annotations on the display unit 12a.

[0079] (Safety training function) Computer 13 determines whether the operator can perform appropriate operations when the construction machinery 4 is in a state that requires attention and may become unsafe. Computer 13 (more specifically, operation determination unit 13b5) determines the appropriateness of the operation received by input device 11, for example, based on the state of construction machinery 4 and the surrounding environment.

[0080] Computer 13 (specifically, object processing unit 13b1) can cause the construction machinery 4 to enter an unsafe state by moving at least one of the objects in the surrounding environment. That is, computer 13 can change the construction machinery 4 from a safe state to an unsafe state by moving at least one of the objects in the surrounding environment. For example, computer 13 can also forcibly implement an unsafe state pre-stored in storage unit 13a by automatically moving at least one of the objects in the surrounding environment. Alternatively, computer 13 can also reproduce a state requiring attention before the construction machinery 4 enters an unsafe state by configuring the objects in the surrounding environment; that is, a state in which the construction machinery 4 might enter an unsafe state due to the operator's input (manual operation) to input device 11. This state requiring attention is, for example, pre-stored in storage unit 13a. In this case, the computer 13 can also determine whether the construction machinery 4 changes to an unsafe state during the movement, while making the object of the construction machinery 4 move in response to the manual operation of the operator in that state, after reproducing the stored state requiring attention.

[0081] Specifically, one example of an unsafe condition is that the construction machinery 4 is operating on an unstable footing, causing the footing to collapse. Another example is that the construction machinery 4 is suspending a work object with a weight exceeding its capacity, resulting in instability and overturning. Yet another example is that the construction machinery 4 suddenly changes its posture on an incline, causing it to shake violently and overturn. Yet another example is that accessories 43 of the construction machinery 4 come into contact with a person approaching the machinery 4. Furthermore, movement includes changes in the posture of the construction machinery 4. For example, movement includes changes in the posture for the construction machinery 4 to perform operations. Operations of the construction machinery 4 include crane operations, operations to excavate work objects, excavation operations of work objects, operations to clamp work objects, operations to hold work objects, operations to crush work objects, and operations to absorb metal work objects.

[0082] When the construction machinery 4 has become unsafe, the computer 13 (specifically the output control unit 13b4) activates the motion seat 12b2 according to the changed unsafe condition. For example, the computer 13 tilts or vibrates the motion seat 12b2 in response to the expected changes in posture or vibration of the construction machinery 4 as it becomes unsafe, thereby informing the operator that the construction machinery 4 has become unsafe.

[0083] When computer 13 reproduces the aforementioned state requiring attention, the operator inputs a safety-enhancing operation into input device 11 based on the condition of at least one of the construction machinery 4 and its surrounding environment. Computer 13 can also provide guidance, prompting the operator to perform an operation that ensures safety. Computer 13 determines the appropriateness of the operation received by input device 11. For example, when input device 11 receives the operation, computer 13 determines whether the operation is appropriate.

[0084] This section provides a specific example of determining the appropriateness of operation based on the conditions of the construction machinery 4 and its surrounding environment. Figure 5 In the example shown, a human object, namely human object 30P, is generated as an object in the surrounding environment, and the situation of human object 30P approaching the construction machinery 4 is reproduced. At this time, a display indicating that the human is approaching the construction machinery 4 is displayed on the display unit 12a1 (in Figure 5 In this example, the central main display is 12a1A. The computer 13 calculates the distance R between the construction machinery 4 and the human object 30P during operation, and determines the appropriateness of the operation based on the calculated distance R. In this embodiment, the computer 13 calculates the distance R from the rotation center of the construction machinery 4 (the rotation axis of the upper rotating body 41b) to the human object 30P.

[0085] The appropriate operation varies depending on the distance R. For example, if the distance R is greater than the reference distance R1 obtained by adding the margin distance α to the maximum reach distance R2 of the construction machinery 4, it is more appropriate not to operate the locking device 11b or the horn operating unit 11c. For example, if the distance R is greater than or equal to the maximum reach distance R2 of the construction machinery 4 but less than the reference distance R1, it is more appropriate to press the horn operating unit 11c to sound the horn. For example, if the distance R is less than the maximum reach distance R2 of the construction machinery 4, it is more appropriate to operate the horn operating unit 11c to sound the horn and unlock the locking device 11b to lock the operation of the construction machinery 4. The appropriate operation for each case is, for example, pre-stored in the storage unit 13a. The computer 13 determines the appropriateness of the operation based on a comparison between the appropriate operation as described above, which varies according to the distance R, and the operation actually input by the operator in the input device 11. That is, when the construction machinery 4 is in a state requiring attention, after the operator inputs an operation into the input device 11, the computer 13 calculates the interval R between the input and the operation, and determines an appropriate operation that matches the calculated interval R by retrieving data from the storage unit 13a, etc. Then, if the determined appropriate operation matches the operation actually input into the input device 11, the operation is deemed appropriate. In other words, the computer 13 determines the appropriateness of the operation based on the content of the operation input into the input device 11 and the interval R between the input and the operation.

[0086] Here, the aforementioned distance R is a safety indicator that affects operational safety. The safety indicator is a value that varies depending on the condition of at least one of the construction machinery 4 and its surrounding environment, and is used as an indicator to determine the appropriateness of the operation.

[0087] Furthermore, although not illustrated, the appropriateness of the operation can be determined based on the crane operation status of the construction machinery 4. For example, the computer 13 calculates the overturning moment of the construction machinery 4 during crane operations to lift goods as the aforementioned safety indicator, and determines the appropriateness of the operation based on the calculated overturning moment. The overturning moment is calculated based on the mass of the goods and the distance from the goods to the overturning fulcrum of the construction machinery 4. For example, if an operation to lift goods by the auxiliary device 43 is input to the input device 11 even though the overturning moment has exceeded a predetermined threshold (in the case of crane operations), the computer 13 determines that the operation is inappropriate.

[0088] After determining the appropriateness of an operation, computer 13 provides guidance to the operator. Computer 13 points out key safety considerations to the operator. One method of providing guidance is to suggest appropriate operation comments to the operator. In this case, various comments corresponding to the determination results are pre-stored in storage unit 13a. After determining the appropriateness of an operation, computer 13 retrieves the comments corresponding to the determination result from storage unit 13a and displays them to the operator. The comments are displayed to the operator, for example, on display unit 12a. Alternatively, the operator can be shown the score already obtained instead of the comments. For example, a score can be calculated based on a pre-determined formula, adding points according to the appropriateness of the operation.

[0089] Computer 13 can also provide instructions to the operator by outputting reports. For example, computer 13 outputs a report containing the hourly status of at least one of the construction machinery 4 and its surrounding environment, and information indicating the appropriateness of the input. Alternatively, computer 13 outputs a report containing the timing of the actual input operation in input device 11 and the appropriate timing for the input operation. Computer 13 stores the reports in storage unit 13a. Computer 13 may also store the reports in storage unit 13a of server device 20.

[0090] (Menu screen) The aforementioned driving instruction and safety training functions are achieved through, for example... Figure 6The menu screen 120 shown is executed. Menu screen 120 is displayed on display unit 12a1. Menu screen 120 is an interface for selecting various functions, including driving instruction and safety training functions. Functions are selected via input device 11. For example, functions can be selected via touchscreen, operating device 11a, or voice input. Specifically, menu screen 120 includes a story mode button 121 labeled "Story Mode". Menu screen 120 includes a driving instruction button 122 labeled "Driving Instruction". Menu screen 120 includes a safety training button 123 labeled "Safety Training". Menu screen 120 includes a field operation button 124 labeled "Field Operation Time Challenge". Menu screen 120 includes a score history button 125 labeled "Score History". After selecting the driving instruction button 122, the driving instruction function (driving instruction mode) is executed. After selecting the safety training button 123, the safety training function (safety training mode) is executed. After selecting the "On-site Operation" button 124, the on-site operation mode is executed. The on-site operation mode allows free operation of the construction machinery 4 at a simulated work site, or a competition based on scores achieved within a specified time. After selecting the "Story Mode" button 121, all driving instruction functions, safety training functions, and the on-site operation mode (story mode) are executed sequentially. After selecting the "Score History" button 125, the score history in the on-site operation mode is displayed (not shown). Figure 7 As shown, for example in the on-site operation mode, the construction machinery 4 can work freely in a virtual space equipped with terrain and other objects. Furthermore, although not shown, a button for selecting the mode for remotely operating the construction machinery 40 can be added to the menu screen 120.

[0091] (Specific examples of computer processing) Computer 13 is capable of performing various processes in multiple sequences. The following is an example illustrating the processing flow of computer 13.

[0092] (Menu processing) Reference Figure 8 Menu processing instructions. Computer 13 displays as follows: Figure 6The menu screen 120 is shown (S10). The computer 13 determines the selected mode (S20). If the story mode is selected, the computer 13 sequentially executes the driving instruction process (S30), safety training process (S40), and on-site operation process (S50). If the driving instruction mode is selected, the computer 13 executes the driving instruction process (S60). If the safety training mode is selected, the computer 13 executes the safety training process (S70). If the on-site operation mode is selected, the computer 13 executes the on-site operation process (S80). Although not shown, if various modes are prepared for multiple types of sites, the computer 13 can also perform processes such as allowing the operator to select a preferred site from these multiple types of sites. After executing each mode, the computer 13 confirms whether to end the menu processing (S90). The computer 13 can allow the operator to confirm whether to end the menu processing, or it can automatically determine whether to end the menu processing. If the determination result in step S90 is "yes", the computer 13 ends the processing. If the determination result in step S90 is "No", computer 13 returns the processing to the menu display in step S10. When multiple styles are prepared for each mode of driving instruction, safety training, etc., the user can select or randomly choose the style of each mode. Additionally, modes that have not been implemented in the past can be selected based on the user's learning records (reports).

[0093] (Driving instruction processing) Reference Figure 9 Explanation of driving instruction processing. Driving instruction processing is handled through the menu (see...). Figure 8 The processing of steps S30 and S60 executed in the process.

[0094] Computer 13 displays a summary description (S100). For example, the summary description includes an overview of the items being implemented in driving instruction. Computer 13 obtains the current posture of the construction machinery 4, such as the angle of the object being operated (S110). For example, if the implemented item is training the forward and backward movement of the stick 43a2, computer 13 obtains the angle of the stick 43a2 relative to the boom 43a1. Computer 13 determines the direction of movement of the construction machinery 4 (S120). Computer 13, for example, determines the target of the stick 43a2 (the movable object) based on the angle of the stick 43a2. Specifically, computer 13 determines whether to rotate the stick 43a2 by +30 degrees or -30 degrees from the current angle of the stick 43a2. Computer 13 displays the target object on display unit 12a1 (S130). The target object is, for example, an object representing the stick 43a2 when it has been moved to the target, or an object representing the auxiliary device operating lever 11a1 when it has been operated to the amount necessary to move the stick 43a2 to the target. The computer 13 displays operating instructions (S140). The computer 13 displays, for example, an instruction indicating which element of the input device 11 should be operated on the display unit 12a1. The computer 13 determines whether the indicated element of the input device 11 (e.g., the auxiliary device operating lever 11a1) has reached the position of the target object (S150). If the determination result in step S150 is "yes," that is, if the indicated element of the input device 11 has reached the position of the target object, the computer 13 provides feedback to the operator (S160). Then, the computer 13 determines whether there is a next item (S170). If the determination result in step S170 is "yes," that is, if there is a next item, the computer 13 returns the process to step S110. If the determination result in step S170 is "no", that is, if there is no next item, the computer 13 ends the processing.

[0095] On the other hand, if the determination result of step S150 is "No," that is, if the element of the indicated input device 11 has not reached the position of the target object, the computer 13 determines whether the input device 11 has been input with a quantity or more of an operation different from the instruction that makes it impossible for the object (movable body) such as the boom 43a2 to reach the target object (S180). If the determination result of step S180 is "Yes," that is, if a quantity or more of an operation different from the instruction has been input, the computer 13 provides feedback to the operator (S190) and returns the process to step S150. If the determination result of step S180 is "No," that is, if no operation different from the instruction has been input, the computer 13 determines whether the input device 11 has not received any input within a specified time (S200). If the determination result of step S200 is "Yes," that is, if no input has been received within a specified time, the computer 13 provides feedback to the operator (S210) and returns the process to step S150.

[0096] (Safety training and handling) Reference Figure 10 Instructions for safety training procedures. Safety training procedures are accessed through the menu (see [reference]). Figure 8 The processing of steps S40 and S70 executed in the process.

[0097] Computer 13 determines the safety training menu (S300). Subsequently, it recreates the state where the construction machinery 4 may become unsafe, i.e., a state requiring attention. In this state, computer 13 begins to move an object (S310). Computer 13, for example, moves a person or object 30P close to the construction machinery 4. Computer 13 determines whether the construction machinery 4 has moved (S320). If the determination result of step S320 is "yes," i.e., the construction machinery 4 has moved, computer 13 causes the motion seat 12b2 to vibrate (S330). Computer 13 determines whether there is operator input to the input device 11 (S340). If the determination result of step S340 is "yes," i.e., there is operation input, computer 13 performs operation determination processing (S350). If the determination result of step S340 is "no," i.e., there is no operation input, computer 13 determines whether the input device 11 has not received input within a specified time (S360). Next, computer 13 displays the result (S370). Computer 13 determines whether there is a next item (S380). If the determination result of step S380 is "yes", that is, if there is a next item, computer 13 returns the process to step S300. If the determination result of step S380 is "no", that is, if there is no next item, computer 13 records a report (S390) and ends the process.

[0098] (Operation judgment and processing) Reference Figure 10 Explanation of operational judgment and handling. Operational judgment and handling are based on safety training procedures (refer to...). Figure 10 The processing of step S350 executed in ).

[0099] Computer 13 first determines the relationship between the construction machinery 4 and the surrounding environment when the operator inputs the operation to input device 11. Specifically, computer 13 calculates the distance R between the construction machinery 4 and the person / object 30P at the time of input, and makes a determination related to the calculated distance R (S400). That is, computer 13 compares the distance R with the reference distance R1 and the maximum reachable distance R2.

[0100] Next, the computer 13 determines the appropriateness of the operation based on the aforementioned distance R calculated at the time of input and the content of the operation input to the input device 11. Specifically, if the distance R is greater than the reference distance R1, the computer 13 determines the appropriateness of the operation based on whether the operator has done nothing (S410). That is, if the person is at a position farther from the construction machinery 4 than the reference distance R1, the appropriate measure that the operator can take is to do nothing, that is, not operate the horn operating unit 11c and the locking device 11b. As mentioned above, the reference distance R1 is the value obtained by adding the maximum reach distance R2 of the construction machinery 4 to the margin distance α (R2 + α). The margin distance α can be set to 2m, for example.

[0101] When the distance R is less than the reference distance R1 and greater than the maximum reach distance R2, the computer 13 determines the appropriateness of the operation based on whether the operator has sounded the horn (S420). That is, when a person is at a position less than the reference distance R1 and greater than the maximum reach distance R2 from the construction machinery 4, the appropriate measure that the operator can take is to press the horn operation unit 11c to sound the horn, that is, to use the horn to warn the approaching person.

[0102] When the distance R between the operator and the machinery is less than the maximum reachable distance R2, the computer 13 determines the appropriateness of the operation based on whether the operator has sounded the horn and locked the lever (S430). That is, when a person is less than the maximum reachable distance R2 from the machinery 4, the appropriate action that the operator can take is to sound the horn to warn the person approaching and to unlock the locking device 11b to lock the operation of the machinery 4.

[0103] (Summarize) The effects produced by the simulator system 1 described above are as follows.

[0104] (1) The simulator system 1 includes a computer 13, a display unit 12a, and an input device 11. The computer 13 reproduces the construction machinery 4 with auxiliary devices 43 and its surrounding environment in a virtual space. The display unit 12a displays the construction machinery 4 and its surrounding environment. The input device 11 accepts operations from the operator that causes the construction machinery 4 to work in the virtual space. The computer 13 includes an object processing unit 13b1 and an operation determination unit 13b5. The object processing unit 13b1 can move objects in at least one of the construction machinery 4 and its surrounding environment to make the construction machinery 4 unsafe. The operation determination unit 13b5 determines the appropriateness of the operation input to the input device 11 based on the content of the operation input by the operator to the input device 11 when the construction machinery 4 is in a state requiring attention before it becomes unsafe.

[0105] In the above structure (1), the operator can virtually experience the unsafe state of the construction machinery 4. Furthermore, the appropriateness of the operation can be assessed based on the content of the operator's input to the input device 11 during the state requiring attention before the unsafe state is declared. Thus, the operator can learn the appropriate operations necessary to ensure safety.

[0106] (2) The simulator system 1 also includes a motion seat 12b2 for the operator to sit in. The computer 13 informs the operator that the construction machinery 4 has become unsafe through the operation of the motion seat 12b2.

[0107] In the above structure (2), the operator can be informed that the construction machinery 4 has become unsafe through the operation of the motion seat 12b2. In addition, the operation of the motion seat 12b2 here may be, for example, a change in the tilt angle of the seat, a vibration of the seat, or a combination of both.

[0108] (3) The input device 11 also has the function of remotely operating the actual engineering machinery 40.

[0109] In the above structure (3), the simulator used for operation training can also be used as a remote device for the actual machine. Therefore, the operator can perform actual operations without changing their sense of operation.

[0110] (4) The operation determination unit 13b5 has the function of calculating safety indicators that affect the safety of operation based on the state of at least one of the construction machinery 4 and the surrounding environment. When the construction machinery 4 is in a state that requires attention, the operation determination unit 13b5 determines the appropriateness of the operation based on the content of the operation input by the input device 11 and the safety indicators calculated when the operation is input.

[0111] In the above structure (4), the operation performed by the operator when the construction machinery 4 is in a state requiring attention can be appropriately evaluated based on the safety indicators at the time of input operation.

[0112] (5) For example, the above safety indicator is the distance R between the engineering machinery 4 and a specific object in the surrounding environment (e.g., a person or object 30P).

[0113] When there are specific objects in the surrounding environment around the construction machinery 4, the appropriate operation to be input into the input device 11 will vary depending on the distance R between the construction machinery 4 and the object. For example, if the object in the surrounding environment is a person 30P, the operator needs to distinguish between various operations such as sounding the horn and locking the lever based on the distance R between the construction machinery 4 and the person 30P. In the above structure (5), the distance R between the input operation and the operation is used as the above safety indicator. Therefore, it is possible to accurately determine whether an appropriate operation corresponding to the distance R has been performed based on the distance R and the operation content.

[0114] (6) When the construction machinery 4 is performing crane operations, the operation determination unit 13b5 calculates the overturning moment based on the mass of the cargo and the distance from the cargo to the overturning fulcrum of the construction machinery 4 as the aforementioned safety indicator. In addition, when the input device 11 inputs an operation to lift the cargo in a state where the calculated overturning moment exceeds a predetermined threshold, the operation determination unit 13b5 determines that the operation is inappropriate.

[0115] In the above structure (6), the appropriateness of the operation can be determined based on whether the operator inputs an operation to perform a crane operation that causes the overturning moment to exceed the threshold.

[0116] (7) The simulator system 1 also includes a storage unit 13a, which pre-stores comments corresponding to the judgments made by the operation judgment unit 13b5. After the operation judgment unit 13b5 makes a judgment, the computer 13 prompts the operator with the comments stored in the storage unit 13a.

[0117] In the above structure (7), by providing the operator with comments corresponding to the judgment result, the operator can easily improve errors, etc.

[0118] (8) After the computer 13 makes a determination in the operation determination unit 13b5, it outputs a report containing the hourly status of at least one of the engineering machinery 4 and the surrounding environment (e.g., people and objects 30P) and information indicating the appropriateness of the operation.

[0119] In the above structure (8), for example, the instructor can use the output report to provide appropriate guidance to the operator.

[0120] (9) After the operation determination unit 13b5 makes a determination, the computer 13 outputs a report, which includes the timing of the operation actually input in the input device 11 and the appropriate timing for the operation to be input.

[0121] In the above structure (9), for example, the instructor can use the output report to provide appropriate guidance to the operator.

[0122] (10) The simulator system 1 also includes a server device 20 that is communicatively connected to the computer 13. The computer 13 stores the aforementioned reports on the server device 20.

[0123] In the above structure (10), the report can be saved in the server device 20.

[0124] (Modified example) Various modifications can be made to the above embodiments. For example, multiple examples of the above embodiments (including modifications) can be combined with each other in various ways. For example, changes can also be made. Figure 1 The connections of the constituent elements shown are as follows. For example, the number of constituent elements in the above embodiment can be changed, or a part of the constituent elements may not be provided. The "constituent element" includes constituent elements (objects) in virtual space (the same applies below). For example, the configuration of the constituent elements can also be changed. For example, the inclusion relationship of the constituent elements can also be changed in various ways. For example, a constituent element described as a subordinate constituent element contained in a superior constituent element may not be included in that superior constituent element, or it may be included in other constituent elements. For example, a constituent element described as multiple components or parts that are different from each other may also be set as a single component or part. For example, a constituent element described as a single component or part may also be set as multiple components or parts that are different from each other. For example, the number of constituent elements in the above embodiment may be changed, or a part may be set as a single component or part. For example, the number of constituent elements in the above embodiment may be changed, or a part may be set as a single component or part. Figures 8-11 The flowchart shown may be in the order of steps, or some steps may be omitted. For example, each component may only have a part of its characteristics (function, configuration, shape, operation, etc.).

Claims

1. A simulator system, characterized in that... include: A computer recreates, in a virtual space, engineering machinery with auxiliary devices and the surrounding environment of said engineering machinery; A display unit shows the construction machinery and its surrounding environment; and The input device accepts operations from the operator who commands the construction machinery to operate within the virtual space, wherein... The computer includes: The object handling unit is capable of moving objects in at least one of the construction machinery and the surrounding environment to make the construction machinery unsafe. as well as The operation determination unit determines the appropriateness of the operation input to the input device based on the content of the operation input by the operator to the input device when the construction machinery was in a state requiring attention before it became an unsafe state.

2. The simulator system according to claim 1, characterized in that... Also includes: The operator sits in a dynamic chair. The computer, through the operation of the motion seat, informs the operator that the construction machinery has become unsafe.

3. The simulator system according to claim 1, characterized in that, The input device also functions as a remote operating device for the actual construction machinery.

4. The simulator system according to claim 1, characterized in that, The operation determination unit has the function of calculating safety indicators affecting operational safety based on the state of at least one of the engineering machinery and the surrounding environment. When the state requiring attention is reached, the operation determination unit determines the appropriateness of the operation based on the content of the operation input to the input device and the safety index calculated when the operation is input.

5. The simulator system according to claim 4, characterized in that, The safety indicator is the distance between the engineering machinery and specific objects in the surrounding environment.

6. The simulator system according to claim 4, characterized in that, When the construction machinery is performing crane operations, the operation determination unit calculates an overturning moment based on the mass of the cargo and the distance from the cargo to the overturning fulcrum of the construction machinery as the safety indicator. When the input device is input with an operation to lift the cargo in a state where the calculated overturning moment exceeds a predetermined threshold, the operation is determined to be inappropriate.

7. The simulator system according to claim 4, characterized in that... Also includes: The storage unit pre-stores annotations corresponding to the determinations made by the operation determination unit. After the operation determination unit makes a determination, the computer prompts the operator with the annotation.

8. The simulator system according to claim 4, characterized in that, After the computer makes a determination in the operation determination unit, it outputs a report, which includes the hourly status of at least one of the construction machinery and the surrounding environment, and information indicating the appropriateness of the operation.

9. The simulator system according to claim 4, characterized in that, After the computer makes a determination in the operation determination unit, it outputs a report, which includes the timing of the operation actually entered in the input device and the appropriate timing when the operation should be entered.

10. The simulator system according to claim 8 or 9, characterized in that... Also includes: The server device is communicatively connected to the computer. The computer stores the report on the server device.

Images