Simulation device, control method, and control program

Abstract

To allow a trainee to easily learn the skill of operating a drilling machine.SOLUTION: A simulation device 1 comprises: a simulation operating device 3 that is provided in a drilling machine, and has a plurality of simulation operating parts 31 artificially reproducing a plurality of operating parts used to actuate the drilling machine; a display device 4 that displays an image of a virtual model obtained by virtually constituting the drilling machine; and a control device 6 that causes the display device 4 to display the image of the virtual model, and artificially actuates a portion of the virtual model, in the image of the virtual model, which corresponds to the simulation operating part 31 operated by a user according to the operation of the simulation operating part 31. The control device 6 includes a display control unit that causes the display device 4 to display the state of the portion of the virtual model artificially actuated according to the operation of a specific simulation operating part 31 by using information other than numerical values.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a simulation device, a control method, and a control program.

Background Art

[0002] Conventionally, in a quarry, a limestone mine, etc., a drilling machine has been used to drill blasting holes in a rock mass (see, for example, Patent Document 1). In such a drilling machine, a tool attached to the tip of a rock drill is pressed against a crushing target by a feed mechanism, and a blasting hole is drilled by transmitting the impact force generated by a striking mechanism and the rotational force generated by a rotating mechanism to the crushing target through the tool. And, in such a drilling operation, positioning of a boom and a guide shell, operations of a feed mechanism, a striking mechanism, and a rotating mechanism, operations of a centering device and a rod changer, etc. are performed. That is, in the drilling machine, the operation targets are diverse, and many operation units for operating each operation target are provided.

[0003] And, since many operation units are provided in the drilling machine as described above, it is not easy to acquire the skill of operating the drilling machine. Therefore, it is conceivable to use a simulation device for acquiring the skill of operating the drilling machine without using the actual drilling machine. For example, the simulation device allows a trainer to visually recognize an image of a virtual model (hereinafter referred to as a model image) that virtually configures a drilling machine through a display device such as a head-mounted display worn by the trainer. Further, the simulation device, in accordance with an operation of the trainer on any one of a plurality of pseudo-operation units that pseudo-reproduce a plurality of operation units provided in the drilling machine, pseudo-acts, on a display screen of the display device, a part such as a boom corresponding to the pseudo-operation unit among the model images. Conventionally, as such a simulation device, a simulation device for a work vehicle such as a forklift has been proposed (see, for example, Patent Document 2).

Prior Art Documents

[0004] [Patent Document 1] Japanese Patent Publication No. 2000-303768 [Patent Document 2] Japanese Patent Publication No. 2004-252024 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] However, in the work process of a drilling machine, there are parts that are difficult for the operator to understand the state of in response to their operation of the control unit. An example of a state that is difficult for the operator to understand is the tightening state of the screws when replacing a rod. Furthermore, the inability to grasp the aforementioned conditions in a drilling machine simulation device hinders trainees from acquiring the skills to operate the drilling machine.

[0006] The present invention has been made in view of the above, and aims to provide a simulation device, a control method, and a control program that enable trainees to easily acquire skills in operating a drilling machine. [Means for solving the problem]

[0007] To solve the above-mentioned problems and achieve the objective, the simulation device according to the present invention comprises: a simulation operation device provided on a drilling machine and having a plurality of simulated operation units that simulate a plurality of operation units used to operate the drilling machine; a display device that displays an image of a virtual model that virtually constitutes the drilling machine; and a control device that displays the image of the virtual model on the display device and simulates the operation of a part of the virtual model corresponding to the simulated operation unit operated by the user in response to the operation of the simulated operation unit, wherein the control device includes a display control unit that displays the state of the part of the virtual model that is simulated to operate in response to a specific operation of the simulated operation unit in information other than numerical values ​​on the display device.

[0008] Furthermore, in the simulation apparatus according to the present invention, information other than the numerical values ​​includes color information, and the display control unit displays the state on the display device using the color, and changes the color in accordance with the change in the state.

[0009] Furthermore, in the simulation apparatus according to the present invention, the display control unit causes the display device to display information other than numerical values ​​regarding the state of parts of the virtual model that are not visible from the display screen of the display device.

[0010] Furthermore, in the simulation device according to the present invention, the aforementioned state is the state in which the screw is fastened.

[0011] Furthermore, the control method according to the present invention is a control method executed by a control device of a drilling machine simulation device, and includes a display processing step of displaying on a display device the state of a part of a virtual model that virtually configures the drilling machine and is operated in a simulated manner in response to an operation on a specific simulated operation unit, using information other than numerical values.

[0012] Furthermore, the control program according to the present invention is a control program that causes a computer to execute a display processing step in which it displays on a display device the state of a part of a virtual model that virtually configures a drilling machine, which is operated in a simulated manner in response to an operation on a specific simulated operation part, using information other than numerical values. [Effects of the Invention]

[0013] According to the simulation device, control method, and control program of the present invention, trainees can easily acquire skills in operating a drilling machine. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 shows a drilling machine that is the target of skill acquisition for operation using a simulation device according to the embodiment. [Figure 2] Figure 2 shows the inside of the rod changer. [Figure 3] Figure 3 shows the configuration of the simulation device according to the embodiment. [Figure 4] Figure 4 is a block diagram showing the configuration of the control device. [Figure 5] Figure 5 shows a specific example of a display image generated by the display control unit. [Figure 6] Figure 6 is a flowchart showing the control method. [Figure 7] Figure 7 is a diagram (part 1) illustrating the first specific example of a display image shown on the display unit during the visualization processing (step S2). [Figure 8] Figure 8 is a diagram (part 2) illustrating the first specific example of a display image shown on the display unit during the visualization display process (step S2). [Figure 9] Figure 9 is a diagram (part 3) illustrating the first specific example of a display image shown on the display unit during the visualization display process (step S2). [Figure 10] Figure 10 shows the positional relationship between the clamp model and the rod model shown in Figure 9. [Figure 11] FIG. 11 is a diagram (part 4) for explaining a first specific example of a display image displayed on the display unit in the visualization display process (step S2). [Figure 12] FIG. 12 is a diagram showing the positional relationship between the clamp portion model and the rod model shown in FIG. 11. [Figure 13] FIG. 13 is a diagram (part 5) for explaining a first specific example of a display image displayed on the display unit in the visualization display process (step S2). [Figure 14] FIG. 14 is a diagram showing the positional relationship between the clamp portion model and the rod model shown in FIG. 13. [Figure 15] FIG. 15 is a diagram (part 1) for explaining a second specific example of a display image displayed on the display unit in the visualization display process (step S2). [Figure 16] FIG. 16 is a diagram (part 2) for explaining a second specific example of a display image displayed on the display unit in the visualization display process (step S2).

MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, embodiments for carrying out the present invention (hereinafter, embodiments) will be described with reference to the drawings. Note that the present invention is not limited by the embodiments described below. Further, in the description of the drawings, the same parts are denoted by the same reference numerals.

[0016] 〔Schematic Configuration of Drilling Machine〕 FIG. 1 is a diagram showing a drilling machine 100 that is a target for performing operation skill acquisition by a simulation device 1 according to an embodiment. First, before explaining the configuration of the simulation device 1 according to the present embodiment, the configuration of the drilling machine 100 that is a target for performing operation skill acquisition by the simulation device 1 will be explained. In explaining the drilling machine 100, the “front side” shown below means the left side shown in FIG. 1.

[0017] The drilling machine 100 drills blast holes H (Figure 1) in the rock mass R (Figure 1) according to the operator's instructions. As shown in Figure 1, the drilling machine 100 comprises a traveling carriage 110 having a pair of left and right tracks 111, and a boom 120, a rock drilling unit 130, and a control room 140 mounted on the traveling carriage 110, respectively.

[0018] The boom 120 is mounted on the right-front side of the traveling carriage 110 so as to be able to rotate (hereinafter referred to as "rotate") around a pivot axis (not shown) along the vertical direction, and so as to be able to rotate (hereinafter referred to as "raise and lower") around a luffing axis (not shown) along the horizontal direction (front-rear direction).

[0019] As shown in Figure 1, the rock-drilling unit 130 is attached to the tip of the boom 120. This rock-drilling unit 130 comprises a guide shell 131, a rock drill 132, a tool 133, and a rod changer 134.

[0020] The guide shell 131 is attached to the tip of the boom 120 and is configured to be able to swing (hereinafter referred to as tilt) in the front-to-back direction, swing (hereinafter referred to as swing) in the left-to-right direction, and slide in the up-to-down direction relative to the tip of the boom 120. The rock drill 132 is mounted along the longitudinal direction of the guide shell 131 so as to be able to move forward and backward on the guide shell 131. The rock drill 132 moves forward and backward on the guide shell 131 by means of a feeding mechanism (not shown) provided on the guide shell 131, and is equipped with a known striking mechanism (not shown) and a rotating mechanism (not shown). The tool 133 is attached to the tip of the rock drill 132. As shown in Figure 1, the tool 133 comprises a shank rod 133A attached to the tip of the rock drill 132, a rod 133C connected to the shank rod 133A by a sleeve 133B, and a bit 133D provided at the tip of the rod 133C.

[0021] Figure 2 shows the inside of the rod changer 134. The rod changer 134 is used to extend a spare rod 133C' (Figures 1 and 2) to the rod 133C, and to retrieve the spare rod 133C'. As shown in Figure 2, the rod changer 134 comprises a rod magazine 134A and a clamp section 134B. The rod magazine 134A is the section where the spare rod 133C' is stored. The clamp section 134B is capable of gripping the spare rod 133C' and the rod 133C, and also moves the spare rod 133C' between the rod magazine 134A and the drilling axis.

[0022] Then, the drilling machine 100 drills blast holes H in the object to be crushed, as shown below. In other words, the rock-breaking unit 130 is positioned at a desired location relative to the object to be broken by the rotational and luffing movements of the boom 120 and the tilting, swinging, and sliding movements of the guide shell 131. After this, the tool 133 is pressed against the object to be broken by a feeding mechanism (not shown), and the impact force generated by the striking mechanism (not shown) and the rotational force generated by the rotation mechanism (not shown) are transmitted to the object to be broken via the tool 133 to drill a blast hole H.

[0023] As shown in Figure 1, the control room 140 is located on the left front side of the traveling chassis 110. Inside the control room 140, as shown in Figure 1, there is a driver's seat (not shown) where an operator of the drilling machine 100 sits, and a number of operating parts located on both sides of the driver's seat that are used to operate the drilling machine 100.

[0024] [Outline configuration of the simulation device] Next, we will describe the configuration of the simulation device 1. Figure 3 shows the configuration of the simulation device 1 according to the embodiment. As shown in Figure 3, the simulation device 1 comprises a training seat 2, a simulated operation device 3, a head-mounted display 4, a base station 5, and a control device 6.

[0025] Training seat 2 is the area where a trainee, who is a user of the simulation device 1, sits to acquire skills in operating the drilling machine 100. As shown in Figure 3, the simulated operation device 3 is a simulated reproduction of the operation device (not shown) provided on the drilling machine 100. The simulated operation device 3 has a plurality of simulated operation units 31, each of which simulates a plurality of operation units (not shown) provided on the said operation device. These plurality of simulated operation units 31 are each connected to the control device 6 wirelessly or by wire, and each outputs an operation signal to the control device 6 in response to user operation by a trainee or the like.

[0026] For example, the simulated operation unit 31 includes a first simulated operation unit 31a and a second simulated operation unit 31b (Figure 3). Here, the first simulated operation unit 31a is located on the right side of the training seat 2. On the other hand, the second simulated operation unit 31b is located on the left side of the training seat 2. Specifically, the first simulated operation unit 31a is a simulated operation unit that simulates the operation units that receive feed operations for operating the feed mechanism (not shown) in the drilling machine 100, impact operations for operating the impact mechanism (not shown), and rotation operations for operating the rotation mechanism (not shown). The second simulated operation unit 31b is a simulated operation unit that simulates the operation unit that receives a clamping operation for operating the clamping unit 134B in the rod changer 134.

[0027] The head-mounted display 4 corresponds to the display device according to the present invention and is connected to the control device 6 wirelessly or via a wired connection for communication. The head-mounted display 4 has the appearance of eyeglasses and is worn by the trainee. The head-mounted display 4 is capable of displaying both a real image and a predetermined image corresponding to the field of view of the trainee wearing the head-mounted display 4. The head-mounted display 4 comprises a shooting unit 41, a display unit 42, and a plurality of infrared output units 43 (see Figure 4).

[0028] The imaging unit 41 is a camera that includes an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) that receives incident light and converts it into an electrical signal. The imaging unit 41 also generates an image by capturing an area corresponding to the field of view of the trainee wearing the head-mounted display 4 under the control of the control device 6. The imaging unit 41 then outputs the data of the generated image to the control device 6.

[0029] The display unit 42 is composed of a display using liquid crystal or organic EL (Electro-Luminescence), and is positioned opposite at least one of the left or right eyes of the trainee wearing the head-mounted display 4. The display unit 42 then displays various images under the control of the control device 6.

[0030] Multiple infrared output units 43 are used to determine the position (3D position) and orientation (the gaze direction of the trainee wearing the head-mounted display 4) of the head-mounted display 4. These multiple infrared output units 43 are each positioned at different locations and each emits (irradiates) infrared light.

[0031] The base station 5 is used to detect the position (3D position) and orientation (the gaze direction of the trainee wearing the head-mounted display 4) of the head-mounted display 4. The base station 5 is also connected to the control device 6 wirelessly or via a wired connection. The base station 5 consists of two infrared cameras 51 (Figure 3) that detect infrared light emitted from multiple infrared output units 43. Note that the number of infrared cameras 51 is not limited to two; any other number may be provided. The two infrared cameras 51 then output the infrared image data generated by the capture to the control device 6.

[0032] Figure 4 is a block diagram showing the configuration of the control device 6. For the sake of explanation, only one simulated operation unit 31, one infrared output unit 43, and one infrared camera 51 are shown in Figure 4. The control device 6 controls the operation of the entire simulation device 1. As shown in Figure 4, the control device 6 comprises an input unit 61, a storage unit 62, and a control unit 63. The input unit 61 consists of buttons, switches, touch panels, etc., that accept user operations by trainers, etc., and outputs signals corresponding to such user operations to the control unit 63.

[0033] The storage unit 62 stores various programs executed by the control unit 63 (including the control program according to the present invention), as well as data necessary when the control unit 63 performs processing. Examples of data required when the control unit 63 performs processing include data for the first 3D model (virtual model), data for the second 3D model, and data for related information. The first 3D model is a 3D model that virtually represents the drilling machine 100. The second 3D model is a 3D model that functions as a screen onto which the captured image generated by the imaging unit 41 is projected. These first and second 3D models are 3D models generated by, for example, CAD (Computer-Aided Design) software, and the position and angle of each part of the first and second 3D models are associated in a virtual spatial coordinate system that is aligned with the real spatial coordinate system on which the simulation device 1 is installed.

[0034] The related information is information used in the visualization display processing described later, which is performed by the control unit 63, and is information that associates the state (amount of movement) of a part of the first 3D model corresponding to a specific simulated operation unit 31 with the amount of operation and the operation time of the operation to the simulated operation unit 31.

[0035] The control unit 63 is implemented by a controller such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in the memory unit 62, and controls the operation of the entire simulation device 1. The control unit 63 is not limited to a CPU or MPU; it may also be composed of integrated circuits such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array). As shown in Figure 4, the control unit 63 comprises a first image acquisition unit 631, a second image acquisition unit 632, a calculation unit 633, an operation recognition unit 634, and a display control unit 635.

[0036] The first image acquisition unit 631 acquires the data of the captured image generated by the shooting unit 41. The second image acquisition unit 632 acquires infrared image data generated by each of the two infrared cameras 51.

[0037] The calculation unit 633 calculates the position and orientation of the head-mounted display 4 (the direction of the trainee's gaze while wearing the head-mounted display 4). Specifically, the calculation unit 633 recognizes the arrival time and angle of infrared light that reached each infrared camera 51 from multiple infrared output units 43 based on the data of each infrared image acquired by the second image acquisition unit 632. Then, the calculation unit 633 calculates the position and orientation of the head-mounted display 4 (the direction of the trainee's gaze while wearing the head-mounted display 4) from the arrival time and angle. The operation recognition unit 634 recognizes the simulated operation unit 31 operated by a trainee wearing the head-mounted display 4 by receiving an operation signal.

[0038] The display control unit 635 generates a display image to be displayed on the display unit 42. Figure 5 shows a specific example of the display image F0 generated by the display control unit 635. In Figure 5, the symbol "120M" represents a boom model (first 3D model) that virtually constructs the boom 120. The symbol "3M" represents an operating device model (first 3D model) that virtually constructs an operating device (not shown) that is installed on the drilling machine 100 and has a configuration similar to the simulated operating device 3. Furthermore, the symbol "SC" represents the screen, which is the second 3D model. The symbol "CI" represents the captured image generated by the imaging unit 41. In Figure 5, dots are added to the captured image CI to distinguish it from the model images of the boom model 120M and the operating device model 3M.

[0039] Specifically, the display control unit 635 generates images of each first 3D model (hereinafter referred to as "model images") that are recognized in areas corresponding to the trainee's field of view in the virtual space, based on the position and orientation of the head-mounted display 4 (the direction of the trainee's gaze while wearing the head-mounted display 4) calculated by the calculation unit 633 and the data of the first 3D model stored in the memory unit 62. In the example in Figure 5, the trainee is facing to the right, and the first 3D models recognized in that area are the boom model 120M and the control device model 3M. The display control unit 635 then generates a display image in which the model images of each of the generated first 3D models are arranged. As a result, the display unit 42 displays this display image. When the trainee is facing to the right, the display unit 42 displays display image F0 (Figure 5), in which the model images of the boom model 120M and the control device model 3M are arranged. Furthermore, the display control unit 635 simulates the operation of the part of the first 3D model corresponding to the simulated operation unit 31 recognized by the operation recognition unit 634 in response to the operation on the simulated operation unit 31.

[0040] Here, when the display control unit 635 generates the display image F0, it places a screen SC (Figure 5) onto which the captured image CI (Figure 5) generated by the imaging unit 41 is projected, in a specific area of ​​the display image F0 (the central area in the left-right direction on the lower side). The display control unit 635 also changes the transparency of the screen SC based on the results of the comparison process shown below. The comparison process involves comparing the shortest distance between a reference line and a pre-set switching reference position with a specific threshold. The reference line is a virtual line created based on the position and orientation of the head-mounted display 4 calculated by the calculation unit 633, and extends linearly from a specific position on the head-mounted display 4 according to the trainee's line of sight. One or more switching reference positions are set at positions corresponding to the placement position of the simulated operating device 3 (the placement position of the simulated operating device 3 in the real space coordinate system, and the position of the operating device model 3M (Figure 5) in the virtual space coordinate system).

[0041] Furthermore, the display control unit 635 makes the screen SC transparent if, as a result of the comparison process, the shortest distance exceeds a certain threshold (hereinafter referred to as the first case). An example of this first case is when the area corresponding to the trainee's field of view in the virtual space is in front or relatively above, and the operating device model 3M is not recognized in that area. In the first case, as a result of making the screen SC transparent, the display image shows only the model images of each first 3D model that are recognized in the area corresponding to the trainee's field of view in the virtual space. Note that the display images F1 to F7 shown in Figures 7 to 13, which will be described later, are the display images in the first case. For this reason, the captured image CI is not displayed in these display images F1 to F7. In Figures 7 to 13, the transparent screen SC is represented by a dashed line.

[0042] On the other hand, the display control unit 635 reduces the transparency of the screen SC if, as a result of the comparison process, the shortest distance is below a certain threshold (hereinafter referred to as the second case). An example of this second case is when the operating device model 3M is recognized in an area corresponding to the trainee's field of view in the virtual space. In the second case, as a result of reducing the transparency of the screen SC, the displayed image shows not only the model images of each first 3D model recognized in an area corresponding to the trainee's field of view in the virtual space, but also the captured image CI. Note that the displayed image F0 shown in Figure 5 is the displayed image in the second case. Therefore, the captured image CI is displayed in the displayed image F0.

[0043] Furthermore, the display control unit 635 performs a visualization display process that displays the state of a part of the first 3D model, which is simulated to operate in response to an operation on a specific simulated operation unit 31, on the display unit 42 using information other than numerical values. Further details regarding the visualization and display process will be explained in the "Control Method" section below.

[0044] [Control method] Next, the control method executed by the control device 6 will be described. Figure 6 is a flowchart showing the control method. For the sake of clarity, the following explanation will primarily focus on the visualization and display processing as the control method executed by the control device 6. First, the display control unit 635 constantly monitors whether a part of the first 3D model (hereinafter referred to as the "specific model") that is simulated to operate in response to an operation on a specific simulated operation unit 31 has been recognized in an area corresponding to the trainee's field of view in the virtual space (step S1).

[0045] Then, if the display control unit 635 determines that a specific model has been recognized in an area corresponding to the field of view of the training in the virtual space (step S1: Yes), it executes a visualization display process (step S2: display processing step). Specifically, the display control unit 635 refers to the relevant information stored in the memory unit 62 and recognizes the state (operation amount) of a specific model associated with the operation amount and operation time of a specific simulated operation unit 31 recognized by the operation recognition unit 634. The display control unit 635 then displays the state (operation amount) on the display unit 42 using non-numerical information within a specific range centered on the placement position of the specific model. In this embodiment, information including color is used as the non-numerical information. The display control unit 635 then displays the recognized state (operation amount) of the specific model on the display unit 42 using the corresponding color, and changes the color according to changes in the state.

[0046] [First specific example of a display image shown in the display unit during visualization processing] Next, a first specific example of the display image shown in the display unit 42 during the visualization process (step S2) described above will be explained. Figures 7 to 14 illustrate a first specific example of a display image displayed in the display unit 42 during the visualization display process (step S2). In Figures 7 to 14, the symbol "133AM" represents a shank rod model (first 3D model) that virtually constructs the shank rod 133A. The symbol "133BM" represents a sleeve model (first 3D model) that virtually constructs the sleeve 133B. Furthermore, the symbol "133CM" represents a rod model (first 3D model) that virtually constructs the rod 133C. Furthermore, the symbol "133C'M" represents a spare rod model (first 3D model) that virtually constructs the spare rod 133C'. Furthermore, the symbol "134M" represents a rod changer model (first 3D model) that virtually constructs the rod changer 134. Furthermore, the symbol "134AM" represents a rod magazine model (first 3D model) that virtually constructs the rod magazine 134A. In addition, the symbol "134BM" represents a clamp section model (first 3D model) that virtually constructs the clamp section 134B. Figure 10 shows the positional relationship between the clamp section model 134BM and the rod model 133CM shown in Figure 9. Figure 12 shows the positional relationship between the clamp section model 134BM and the rod model 133CM shown in Figure 11. Figure 14 shows the positional relationship between the clamp section model 134BM and the rod model 133CM shown in Figure 13.

[0047] As described above, the clamp section 134B is used when adding a spare rod 133C' to the rod 133C and when retrieving the spare rod 133C'. However, the state in which the clamp section 134B grips the spare rod 133C' and the rod 133C (the state of the opening angle of the clamp section 134B (hereinafter referred to as the opening state)) is difficult for the operator to see. In other words, it is difficult for the operator to acquire the skills to operate the clamp section 134B (clamp operation). Therefore, in the simulation device 1, as shown in the first specific example in Figures 7 to 14, the state is displayed on the display unit 42 using information other than numerical values ​​in the visualization display process (step S2), thereby enabling the trainee to acquire the skills to operate the clamp on the second simulated operation unit 31b.

[0048] Specifically, if the display control unit 635 determines that the clamp unit model 134BM, which is a specific model, has been recognized in an area corresponding to the field of view of the trainee in the virtual space (step S1: Yes), it executes a visualization display process (step S2). For example, Figure 7 illustrates the display image F1 when the clamp unit model 134BM is located between the rod magazine model 134AM and the drilling axis, and the opening is in a closed state (a state in which the spare rod 133C' and rod 133C can be gripped). In this case, the display control unit 635 recognizes the closed state from the amount and time of the clamp operation on the second simulated operation unit 31b during the visualization display process (step S2), and displays state information SI1 corresponding to the closed state on the display unit 42 at a position close to the clamp unit model 134BM (Figure 7). For example, the state information SI1 consists of an image filled in red to indicate the closed state. Note that in Figure 7, the red is represented by diagonal lines for the sake of explanation.

[0049] Furthermore, for example, Figure 8 illustrates a display image F2 when the clamp unit model 134BM is located between the rod magazine model 134AM and the drilling axis, and the opening is in an open state (a state in which the gripping of the spare rod 133C' and rod 133C is released). In this case, the display control unit 635 recognizes the open state from the amount and time of the clamp operation on the second simulated operation unit 31b in the visualization display processing (step S2), and displays state information SI1 corresponding to the open state on the display unit 42 at a position close to the clamp unit model 134BM (Figure 8). For example, the state information SI1 consists of an image filled with green to indicate the open state. Note that in Figure 8, for the sake of explanation, the green is represented by dots.

[0050] Furthermore, for example, Figure 9 illustrates a display image F3 when the clamp model 134BM is positioned to grip the rod model 133CM located on the drilling axis, and the opening state is open. Specifically, in the state shown in Figure 9, as shown in Figure 10, the opening state of the clamp model 134BM is open. That is, a large gap GA (Figure 10) is provided between the clamp model 134BM and the rod model 133CM, and the grip of the rod model 133CM by the clamp model 134BM is released. In this case, the display control unit 635 displays the same state information SI1 as in Figure 8 on the display unit 42 during the visualization display process (step S2) (Figure 9).

[0051] Furthermore, for example, Figure 11 illustrates a display image F4 when the clamp model 134BM is positioned to grip the rod model 133CM located on the drilling axis, and the opening state is closed. Specifically, in the state shown in Figure 11, as shown in Figure 12, the opening state of the clamp model 134BM is closed. That is, there is virtually no gap GA between the clamp model 134BM and the rod model 133CM, and the rod model 133CM is gripped by the clamp model 134BM. In this case, the display control unit 635 displays the same state information SI1 as in Figure 7 on the display unit 42 during the visualization display process (step S2) (Figure 11).

[0052] Furthermore, for example, Figure 13 illustrates display image F5 when the clamp part model 134BM is positioned to grip the rod model 133CM located on the drilling axis, and the opening state is a half-clamp state (a state in which the spare rod 133C' and rod 133C are supported so as to be rotatable about their central axes). Specifically, in the state shown in Figure 13, as shown in Figure 14, the opening state of the clamp part model 134BM is a half-clamp state. That is, a small gap GA (Figure 14) is provided between the clamp part model 134BM and the rod model 133CM, and the rod model 133CM is supported by the clamp part model 134BM so as to be rotatable about its central axis. In this case, the display control unit 635 recognizes the half-clamp state in the visualization display process (step S2) based on the amount and duration of the clamp operation on the second simulated operation unit 31b, and displays state information SI1 corresponding to the half-clamp state on the display unit 42 at a position close to the clamp unit model 134BM (Figure 13). For example, the state information SI1 consists of an image in which half is filled with red and the other half is filled with green to indicate the half-clamp state. In Figure 13, for the sake of explanation, the red is represented by diagonal lines and the green is represented by dots. Furthermore, the proportions of the red and green areas are not limited to the above proportions and may be other proportions.

[0053] [Second specific example of a display image shown in the display unit during visualization processing] Next, a second specific example of the display image shown in the display unit 42 during the visualization process (step S2) described above will be explained. Figures 15 and 16 illustrate a second specific example of the display image shown in the display unit 42 during the visualization display process (step S2). Specifically, Figures 15 and 16 correspond to Figures 7 through 9, Figure 11, and Figure 13.

[0054] When adding a spare rod 133C' to rod 133C and retrieving the spare rod 133C', operations (feed operation, impact operation, and rotation operation) are performed to activate the feed mechanism (not shown), impact mechanism (not shown), and rotation mechanism (not shown) of the drilling machine 100. This results in tightening and loosening of screws between the shank rod 133A and rod 133C (sleeve 133B), and between rod 133C and spare rod 133C'. However, the operator cannot visually confirm the state of the screw fastening between these parts. In other words, it is difficult for the operator to acquire the skills for the feed operation, impact operation, and rotation operation. Therefore, in the simulation device 1, as shown in the second specific example in Figures 15 and 16, the tightening status of the screw, which is not visible from the display screen of the display unit 42, is displayed on the display unit 42 using information other than numerical values, thereby enabling the trainee to acquire the skills of feeding, striking, and rotating operations on the first simulated operation unit 31a.

[0055] Specifically, if the display control unit 635 determines that the sleeve model 133BM, which is a specific model, has been recognized in an area corresponding to the field of view of the training in the virtual space (step S1: Yes), it executes a visualization display process (step S2). For example, Figure 15 illustrates a display image F6 when the screw fastening state between the shank rod model 133AM and the sleeve model 133BM is at its loosest. In this case, the display control unit 635 recognizes the fastening state from the amount and time of the feed operation and rotation operation to the first simulated operation unit 31a during the visualization display process (step S2), and displays state information SI2 corresponding to the fastening state on the display unit 42 on the outer circumference of the sleeve model 133BM (Figure 15). For example, the state information SI2 consists of an image filled with green to indicate the fastening state where the screw is at its loosest. In Figure 15, for the sake of explanation, the green is represented by dots.

[0056] Furthermore, for example, Figure 16 illustrates a display image F7 when the screw fastening between the shank rod model 133AM and the sleeve model 133BM is tightened to about 50%. In this case, the display control unit 635 recognizes the fastening state from the amount and time of the feed operation and rotation operation to the first simulated operation unit 31a in the visualization display processing (step S2), and displays state information SI2 corresponding to the fastening state on the display unit 42 on the outer circumference of the sleeve model 133BM (Figure 16). For example, the state information SI2 consists of an image in which the amount the screw is tightened is represented by filling in red and the amount the screw is loose is represented by filling in green to indicate the fastening state of about 50%. Note that in Figure 16, for the sake of explanation, the red is represented by diagonal lines and the green is represented by dots.

[0057] Furthermore, in the cases shown in Figures 7 to 9, Figure 11, and Figure 13, the sleeve model 133B, which is a specific model, is recognized in the area corresponding to the trained field of view in the virtual space. Figures 7 to 9, 11, and 13 illustrate display images F1 to F5 when the screw is tightened to its maximum extent between the shank rod model 133AM and the sleeve model 133BM. In this case, the display control unit 635 recognizes the tightening state from the amount and time of the feed operation and rotation operation to the first simulated operation unit 31a during the visualization display process (step S2), and displays state information SI2 corresponding to the tightening state on the display unit 42 on the outer circumference of the sleeve model 133BM (Figures 7 to 9, 11, and 13). For example, the state information SI2 consists of an image that is mostly filled in red and partially filled in green to indicate the tightened state where the screw is tightened to its maximum extent. Note that in Figures 7 to 9, 11, and 13, for the sake of explanation, the red is represented by diagonal lines and the green is represented by dots.

[0058] According to the embodiment described above, the following effects are achieved. In the simulation device 1 according to this embodiment, the control device 6 performs a visualization display process (step S2) in which it displays the state of a part of the first 3D model that is simulated to operate in response to an operation on a specific simulated operation unit 31 on the display unit 42 using information other than numerical values. Therefore, trainees can grasp conditions that are difficult to grasp (for example, the opening state of the clamp portion 134B, the tightening state of the screws between the shank rod 133A and the rod 133C (sleeve 133B), and between the rod 133C and the spare rod 133C') through a visualization display process (step S2). Therefore, according to the simulation device 1 of this embodiment, difficult-to-understand conditions can be made clear to the trainee through visualization and display processing (step S2), and the skills to operate the drilling machine 100 can be easily acquired.

[0059] In particular, the control device 6 displays the above-mentioned state on the display unit 42 using color during the visualization display process (step S2), and changes the color according to the change in the state. Therefore, trainees can easily grasp the change in the state.

[0060] (Other embodiments) While embodiments for carrying out the present invention have been described so far, the present invention should not be limited to the embodiments described above. In the embodiment described above, examples of states to be displayed on the display unit 42 with information other than numerical values ​​by the visualization display process (step S2) include the opening state of the clamp portion 134B and the tightening state of the screws between the shank rod 133A and the rod 133C (sleeve 133B) and between the rod 133C and the spare rod 133C', but the embodiment is not limited to these. If the state in question is difficult for the trainee to recognize, other states may be adopted. For example, as the tightening state of the screws between each of the above-mentioned parts, after the screw is set to the tightest state, the state of the axial force between each of those parts, which changes according to the number of times the striking mechanism (not shown) has operated, may be displayed on the display unit 42 using information other than numerical values.

[0061] In the embodiment described above, the visualization display process (step S2) displays the above-described state in color on the display unit 42, and changes the color in accordance with the change in the state, but the embodiment is not limited to this. Any display format that shows the state using information other than numerical values ​​is acceptable. For example, a display format that shows the state on the display unit 42 using a bar and changes the length of the bar in accordance with changes in the state is also acceptable.

[0062] In the embodiments described above, the captured image CI was displayed on the display unit 42 in the second case so that the trainee could operate multiple simulated operation units 31 while wearing the head-mounted display 4, but the invention is not limited to this. For example, an infrared light-emitting unit is attached to the trainee's hand. Also, data of a 3D model corresponding to a human hand is stored in the memory unit 62. Here, the control unit 63 calculates the position and orientation of the trainee's hand based on the infrared images generated by the two infrared cameras 51, similar to how the position and orientation of the head-mounted display 4 is calculated. Then, based on the calculated position and orientation of the trainee's hand, the control unit 63 displays a model image of the 3D model corresponding to a human hand on the display unit 42.

[0063] In the embodiment described above, some functions of the control unit 63 may be provided outside the control device 6. For example, the functions of the calculation unit 633, which calculates the position and orientation of the head-mounted display 4, may be provided on the base station 5. In this case, the control device 6 acquires information indicating the position and orientation of the head-mounted display 4 from the base station 5.

[0064] In the visualization display process (step S2) according to the above-described embodiment, the above-described state may be displayed numerically on the display unit 42. [Explanation of Symbols]

[0065] 1. Simulation device 2 training seats 3 Simulated operation device 3M Control Device Model 4. Head-mounted display 5 Base Stations 6. Control device 31 Simulation operation section 31a First simulated operating unit 31b Second simulated operating unit 41 Photography Department 42 Display section 43 Infrared output section 51 Infrared Camera 61 Input section 62 Storage section 63 Control Unit 100 Drilling Machine 110 Bogie 111 Trucks 120 Boom 120M Boom Model 130 rock-cutting units 131 Guide Shell 132 Rock drilling machine 133 Tools 133A Shank Rod 133AM Shank Rod Model 133B Sleeve 133BM Sleeve Model 133C Rod 133CM Rod Model 133C' Spare rod 133C'M Spare Rod Model 133D bits 134 Rod Changer 134M Rod Changer Model 134A Rod Magazine 134AM Rod Magazine Model 134B Clamp section 134BM Clamp Model 140 Cockpit 631 First image acquisition unit 632 Second image acquisition unit 633 Calculation Unit 634 Operation recognition section 635 Display Control Unit CI (Civil Identity) Photographed Images F0~F7 Display Images H blast hole R bedrock SC Screen SI1, SI2 Status Information

Claims

1. A simulated operating device having multiple simulated operating parts that are provided on a drilling machine and each simulated reproduces one of the multiple operating parts used to operate the drilling machine, A display device that displays an image of a virtual model of the drilling machine, The system includes a device that displays an image of the virtual model on the display device, and a control device that simulates the operation of a part of the virtual model corresponding to the simulated operation unit operated by the user in response to the operation of the simulated operation unit. The control device is A simulation apparatus comprising a display control unit that displays on a display device the state of a part of a virtual model that is operated in a simulated manner in response to an operation on a specific simulated operation unit, the state of a specific model associated with the amount of operation and the operation time of the operation, using information other than numerical values.

2. Information other than the aforementioned numerical values ​​is, This is information that includes color. The display control unit, The simulation apparatus according to claim 1, wherein the aforementioned state is displayed on the display device by the aforementioned color, and the color is changed in accordance with the change in the said state.

3. The display control unit, The simulation apparatus according to claim 1 or 2, wherein the state of a part of the virtual model that is not visible from the display screen of the display device is displayed on the display device by information other than the numerical value.

4. The aforementioned state is, The simulation device according to claim 3, which represents the state of screw fastening.

5. A control method performed by the control device of a drilling machine simulation device, A control method comprising a display processing step of displaying on a display device the state of a part of a virtual model that virtually configures the drilling machine, which is operated in a simulated manner in response to an operation on a specific simulated operation unit, and which is associated with the amount of operation and the operation time of the operation, using information other than numerical values.

6. A control program that causes a computer to execute a display processing step, which displays on a display device the state of a part of a virtual model that virtually configures a drilling machine, which is operated in a simulated manner in response to an operation on a specific simulated operating part, and which is associated with the amount of operation and the operation time of the said operation, using information other than numerical values.

Images