Information provision device, information provision method, and information provision program

Abstract

To improve convenience.SOLUTION: An information provision device 7 comprises a determination unit 712 that determines an operation unit that a user intends to operate from among the plurality of operation units used to operate a drilling machine as a target operation unit, and an information provision unit 713 that provides operation content information indicating the operation content of the target operation unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information providing device, an information providing method, and an information providing program.

Background Art

[0002] Conventionally, drilling machines have been used in quarries, limestone mines and the like to drill blast holes in rock masses (for example, refer to Patent Document 1). In such a drilling machine, a tool mounted at the tip of a rock drill is pressed against the object to be crushed by a feed mechanism, and the impact force generated by the striking mechanism and the rotational force generated by the rotating mechanism are transmitted to the object to be crushed via the tool, thereby drilling the blast hole. In such drilling operations, positioning of the boom and guide shell, operation of the feed mechanism, striking mechanism and rotating mechanism, operation of the centralizer and rod changer, and the like are performed. That is, the drilling machine has a wide variety of operation targets, and is provided with a large number of operation sections for operating each of the operation targets.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, it is not easy to fully master the operation content of each operation section (the content indicating the operation target corresponding to the operation section, the operating state of the operation target according to the operation on the operation section, etc.) for every operation section. Here, it is conceivable to provide a pictogram indicating the operation content for each operation section on the panel where the operation sections are arranged. However, the space for providing pictograms on the panel is not sufficient. As a result, when pictograms are provided on the panel, the pictograms become small and are difficult to visually recognize. Therefore, there is a need for technology that can improve convenience by providing the user with information indicating the operation content of the control unit they are trying to operate.

[0005] The present invention has been made in view of the above, and aims to provide an information provision device, an information provision method, and an information provision program that can improve convenience. [Means for solving the problem]

[0006] To solve the above-mentioned problems and achieve the objective, the information providing device according to the present invention comprises a determination unit that determines which of a plurality of operating units used to operate a drilling machine the operating unit that the user intends to operate is the target operating unit, and an information providing unit that provides operation content information indicating the operation content of the target operating unit.

[0007] Furthermore, in the information providing device according to the present invention, the determination unit determines that, among the plurality of operation units, the operation unit whose positional relationship with the position of the user's hand is a specific positional relationship is the target operation unit.

[0008] Furthermore, in the information providing device according to the present invention, the determination unit measures the position coordinates of the user's hand, analyzes the positional relationship between the position coordinates of the user's hand and the pre-set position coordinates of the plurality of operation units, and determines that the operation unit that has the specific positional relationship is the target operation unit based on the results of the analysis.

[0009] Furthermore, the information providing device according to the present invention further includes an image acquisition unit that acquires data of a captured image of an area including the plurality of operation units, and the determination unit measures the position coordinates of the user's hand based on the captured image, analyzes the positional relationship between the position coordinates of the user's hand and the pre-set position coordinates of the plurality of operation units, and determines that the operation unit that has the specific positional relationship is the target operation unit based on the result of the analysis.

[0010] Furthermore, in the information providing device according to the present invention, the determination unit recognizes the position of the user's hand by detecting the skeletal structure of the user's hand based on the captured image.

[0011] Furthermore, in the information providing device according to the present invention, the determination unit determines that the operation unit among the plurality of operation units is the target operation unit whose position coordinates in the specific positional relationship are shortest and whose distance from the position coordinates of the user's hand is below a specific threshold.

[0012] Furthermore, the information provision method according to the present invention is an information provision method performed by an information provision device, and includes a determination step of determining which of a plurality of operating units used to operate a drilling machine is an operating unit that a user intends to operate, and an information provision step of reading operation content information indicating the operation content of the target operating unit from a storage unit and providing it.

[0013] Furthermore, the information provision program according to the present invention is an information provision program that causes a computer to execute a determination step of determining which of the multiple operating units used to operate a drilling machine the user intends to operate is the target operating unit, and an information provision step of reading operation content information indicating the operation content of the target operating unit from a storage unit and providing it. [Effects of the Invention]

[0014] According to the information provision device, information provision method, and information provision program of the present invention, convenience can be improved. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 is a diagram showing a drilling machine according to an embodiment. [Figure 2] Figure 2 shows the interior of the cockpit. [Figure 3] Figure 3 is a block diagram showing the configuration of the operation support device according to the embodiment. [Figure 4] Figure 4 is a diagram illustrating the related information stored in the related information database. [Figure 5] Figure 5 is a flowchart illustrating the information providing method according to the embodiment. [Figure 6] Figure 6 is a diagram (Part 1) for explaining the first specific example of the information providing method. [Figure 7] Figure 7 is a diagram (Part 2) for explaining the first specific example of the information providing method. [Figure 8] Figure 8 is a diagram (Part 3) for explaining the first specific example of the information providing method. [Figure 9] Figure 9 is a diagram (Part 4) for explaining the first specific example of the information providing method. [Figure 10] Figure 10 is a diagram (Part 5) for explaining the first specific example of the information providing method. [Figure 11] Figure 11 is a diagram (Part 1) for explaining the second specific example of the information providing method. [Figure 12] Figure 12 is a diagram (Part 2) for explaining the second specific example of the information providing method. [Figure 13] Figure 13 is a diagram (Part 3) for explaining the second specific example of the information providing method. [Figure 14] Figure 14 is a diagram (Part 4) for explaining the second specific example of the information providing method. [Mode for Carrying Out the Invention]

[0016] Hereinafter, modes for carrying out the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings. Note that the present invention is not limited by the embodiments described below. Furthermore, in the descriptions of the drawings, the same reference numerals are assigned to the same parts.

[0017] [General Configuration of Drilling Machine] Figure 1 is a diagram illustrating the drilling machine 1 according to the embodiment. ​The drilling machine 1 is used to drill blasting holes in rock. This drilling machine 1 comprises a drilling machine body 2 (Figure 1) that drills blasting holes in rock according to the operation of the user, an operator, and an operation support device 3 (see Figure 3) that assists the operator's operation. As shown in Figure 1, the drilling machine body 2 comprises a traveling carriage 21 having a pair of left and right tracks 211 (only one track 211 is shown in Figure 1), and a boom 22, a rock-drilling unit 23, and a control room 24 mounted on the traveling carriage 21, respectively.

[0018] The boom 22 is mounted on the right-front side of the traveling carriage 21 so as to be able to rotate (rotate) around a pivot axis 221 (Figure 1) that is aligned vertically, and so as to be able to rotate (raise and lower) around a luffing axis 222 (Figure 1) that is aligned horizontally (forward and backward).

[0019] As shown in Figure 1, the rock-drilling unit 23 is attached to the tip of the boom 22. This rock-drilling unit 23 comprises a guide shell 231 attached to the tip of the boom 22, a rock-drilling machine 232 that is provided to move forward and backward along the longitudinal direction of the guide shell 231, and a tool 233 attached to the tip of the rock-drilling machine 232. Here, the rock drill 232 moves forward and backward on the guide shell 231 by a feed mechanism (not shown) provided on the guide shell 231, and is equipped with a known striking mechanism (not shown) and a rotation mechanism (not shown). Although not specifically shown, the tool 233 consists of a shank rod, sleeve, rod, and bit connected in this order from the base end. Then, in the drilling machine body 2, the tool 233 is pressed against the object to be crushed by a feeding mechanism (not shown), and the impact force generated by the striking mechanism (not shown) and the rotational force generated by the rotation mechanism (not shown) are transmitted to the object to be crushed via the tool 233 to drill a blast hole.

[0020] As shown in Figure 1, the control room 24 is located on the left-front side of the traveling carriage 21. Inside the control room 24, as shown in Figure 1, there is an operator's seat 241 where the operator who operates the drilling machine body 2 sits. Figure 2 shows the interior of the cockpit 24. As shown in Figure 2, within the cockpit 24, an operating device 25 is provided to the right of the driver's seat 241, as shown in Figure 2.

[0021] The operating device 25 is operated by an operator seated in the driver's seat 241. The operating panel 251 of this operating device 25 is equipped with multiple operating sections 252a to 252r (Figure 2) used to operate the drilling machine body 2. Of these operating sections 252a to 252r, operating section 252a is the drilling operating section, and operating section 252b is the boom operating section. The drilling operation unit 252a is an operation unit that operates the feed mechanism (not shown), the impact mechanism (not shown), and the rotation mechanism (not shown), respectively. This drilling operation unit 252a includes a control grip for operating the feed mechanism (not shown), an impact switch for operating the impact mechanism (not shown), and a rotation switch for operating the rotation mechanism (not shown). In Figure 2, for the sake of explanation, only the control grip is shown as part of the drilling operation unit 252a. The boom control unit 252b is an operating unit that performs the rotational movement of the boom 22 around the pivot axis 221 and the luffing movement around the luffing axis 222, respectively.

[0022] The operation support device 3 is located inside the cockpit 24 and is a device that assists the operator in operating the control device 25. The following describes the detailed configuration of the operation support device 3.

[0023] [Configuration of the operation support device] Figure 3 is a block diagram showing the configuration of the operation support device 3. As shown in Figure 3, the operation support device 3 comprises a shooting device 4, a notification device 5, an input device 6, and an information providing device 7. The imaging device 4 is a camera that includes an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) that receives incident light and converts it into an electrical signal, and captures a specific area to generate an image. As shown in Figure 2, the imaging device 4 is positioned above the operation panel 251 and, under the control of the information providing device 7, captures an area including multiple operation units 252a to 252r to generate an image. The imaging device 4 then outputs the data of the generated image to the information providing device 7.

[0024] As shown in Figure 3, the notification device 5 includes a display unit 51 and an audio output unit 52. The display unit 51 is composed of a display display using liquid crystal or organic EL (Electro-Luminescence), and displays various images under the control of the information providing device 7. For example, as shown in Figure 2, the display unit 51 is positioned above the operating device 25. The audio output unit 52 includes a speaker 521 and converts the digital audio signal input from the information providing device 7 into an analog audio signal by D / A (Digital / Analog) conversion, and outputs audio corresponding to the analog audio signal from the speaker 521. For example, as shown in Figure 2, the audio output unit 52 is provided integrally with the display unit 51.

[0025] The input device 6 consists of buttons, switches, touch panels, etc., that accept user operations from an operator, and outputs operation signals corresponding to such user operations to the information providing device 7. Some or all of the functions of the input device 6 may be arranged as a touch panel on the surface of the display unit 51.

[0026] The information providing device 7 provides the operator with operation content information, which indicates the operation content of the operation unit that the operator intends to operate from among the multiple operation units 252a to 252r, via the notification device 5. As shown in Figure 3, this information providing device 7 comprises a control unit 71 and a storage unit 72.

[0027] The control unit 71 is implemented by a controller such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs (including the information provision program according to the present invention) stored in the storage unit 72, and controls the operation of the entire information provision device 7. Note that the control unit 71 is not limited to a CPU or MPU, but may also be composed of an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array). As shown in Figure 3, the control unit 72 includes an image acquisition unit 711, a determination unit 712, and an information provision unit 713.

[0028] The image acquisition unit 711 acquires the data of the captured image generated by the imaging device 4. Based on the captured image, the determination unit 712 determines which of the multiple operation units 252a to 252r the operator intends to operate as the target operation unit. The information provision unit 713 reads operation content information from the storage unit 72, which indicates the operation content of the target operation unit (such as the target operation corresponding to the target operation unit and the operating status of the target operation in response to the operation on the target operation unit), outputs it to the notification device 5, and provides it to the operator via the notification device 5.

[0029] The storage unit 72 stores various programs executed by the control unit 71 (information provision programs according to the present invention), as well as data necessary when the control unit 71 performs processing. This storage unit 72 includes a related information DB (Data Base) 721 and a learning model DB (Data Base) 722.

[0030] Figure 4 is a diagram illustrating the related information stored in the related information DB721. Specifically, Figure 4 shows the main part of the captured image F0 generated by the imaging device 4. Here, as shown in Figure 4, the position coordinates Pa to Pr of the multiple operating units 252a to 252r included as subjects in the captured image F0 are each predetermined position coordinates. In this embodiment, these position coordinates are the pixel positions (x,y) in the captured image F0, and are two-dimensional coordinates in the horizontal plane. The related information DB721 stores related information in which each position coordinate Pa to Pr is associated with the operation content information of the operating units 252a to 252r corresponding to each position coordinate Pa to Pr. For example, operation content information indicating the operation content of the drilling operating unit 252a is associated with the position coordinate Pa. Also, operation content information indicating the operation content of the boom operating unit 252b is associated with the position coordinate Pb. This operation content information consists of text data, audio data, or image data.

[0031] The learning model DB722 stores the learning model used in image recognition performed by the determination unit 712. This learning model is, for example, a model generated by machine learning using artificial intelligence (AI). Further details about the learning model will be explained in the "Information Provision Method" section below.

[0032] [Information provision method] Next, we will describe the information provision method performed by the information provision device 7. Figure 5 is a flowchart showing the methods of providing information. First, the image acquisition unit 711 acquires the captured image generated by the imaging device 4 (step S1).

[0033] After step S1, the determination unit 712 uses image recognition (so-called skeleton detection) using the learning model stored in the learning model DB 722 to detect the skeleton of the worker's hand included as a subject in the captured image acquired in step S1, thereby recognizing the position coordinates of the hand (pixel position (x,y) in the captured image) (step S2). In this embodiment, the determination unit 712 recognizes the position coordinates of the position between the thumb and index finger of the hand as the position coordinates of the hand. The learning model stored in the DB722 learning model is obtained by using images of human hands that have been photographed and pre-labeled with the positions of the joint points of the hand as training data, and then using machine learning (e.g., deep learning) to determine the positions of the joint points based on this training data.

[0034] After step S2, the determination unit 712 determines the target operation unit (step S3). Specifically, the determination unit 712 analyzes the positional relationship between the position coordinates of the worker's hand recognized in step S2 and each position coordinate Pa to Pr included in the related information stored in the related information DB 721. Then, the determination unit 712 determines that the operation unit among the multiple operation units 252a to 252r whose positional relationship with the position coordinates of the worker's hand recognized in step S2 is shortest and whose distance from that position coordinate is below a certain threshold is determined to be the target operation unit that the worker intends to operate. Steps S1 to S3 described above correspond to the determination steps according to the present invention.

[0035] After step S3, the information provision unit 713 refers to the related information stored in the related information DB 721 and provides the operator with operation content information associated with the position coordinates of the target operation unit determined in step S3 via the notification device 5 (step S4: information provision step). As a result, the operator recognizes the operation content of the target operation unit they intend to operate by receiving the operation content information notification from the notification device 5. The information provision method performed by the information provision device 7 is completed by the above steps S1 to S4.

[0036] [First example] Next, we will explain the first specific example of the information provision method described above. Figures 6 to 10 illustrate a first specific example of the information provision method. Specifically, Figure 6 corresponds to Figure 4 and shows the main part of the captured image F1 generated by the imaging device 4. Figures 7 to 10 show the operation content information Ia1 to Ia4 displayed on the display unit 51, respectively.

[0037] The first specific example corresponds to the case where the worker's hand U, which is included as a subject in the captured image F1, is in the position shown in Figure 6. In this case, in step S2, the determination unit 712 recognizes the position coordinate SP (pixel position (x,y) in the captured image F1) of the position between the thumb TH and index finger IF of the hand U as the position coordinate of the hand U. Furthermore, in step S3, the determination unit 712 determines that the drilling operation unit 252a with the shortest position coordinate Pa among the multiple operation units 252a to 252r is the target operation unit because the distance from the position coordinate SP of the worker's hand U is less than or equal to a specific distance.

[0038] Then, in step S4, the information provision unit 713 displays some or all of the operation content information Ia1 to Ia4 (Figures 7 to 10) indicating the operation content of the drilling operation unit 252a (target operation unit) on the display unit 51. Specifically, operation information Ia1, as shown in Figure 7, is information that describes the positions of the control grip, impact switch, and rotary switch in the drilling operation unit 252a using images and text. Operation information Ia2, as shown in Figure 8, is information that describes the operating state of the target being operated on in response to the operation of the control grip in the drilling operation unit 252a using images and text. Operation information Ia3, as shown in Figure 9, is information that describes the operating state of the target being operated on in response to the operation of the rotary switch in the drilling operation unit 252a using images and text. Operation information Ia4, as shown in Figure 10, is information that describes the operating state of the target being operated on in response to the operation of the impact switch in the drilling operation unit 252a using images and text.

[0039] Here, the information providing unit 713 sequentially switches the operation content information Ia1 to Ia4 displayed on the display unit 51 in response to a press of the "next" or "back" icons IC1 and IC2 (Figures 7 to 10), which correspond to the input device 6 composed of a touch panel, or according to the elapsed time. Furthermore, the information providing unit 713 terminates the display of operation content information Ia1 to Ia4 on the display unit 51 in response to a press of the "end" icon IC3 (Figures 7 to 10), which corresponds to the input device 6 composed of a touch panel. Furthermore, the configuration is not limited to sequentially switching between displaying the operation information Ia1 to Ia4 on the display unit 51; it is also acceptable to configure the system to display all of the operation information Ia1 to Ia4 on the display unit 51.

[0040] [Second specific example] Figures 11 to 14 illustrate a second specific example of the information provision method. Specifically, Figure 11 corresponds to Figure 4 and shows the main part of the captured image F2 generated by the imaging device 4. Figures 12 to 14 show the operation content information Ib1 to Ib3 displayed on the display unit 51, respectively.

[0041] The second specific example corresponds to the case where the worker's hand U, which is included as a subject in the captured image F2, is in the position shown in Figure 11. In this case, in step S2, the determination unit 712 recognizes the position coordinate SP (pixel position (x,y) in the captured image F2) of the position between the thumb TH and index finger IF of the hand U as the position coordinate of the hand U. Furthermore, in step S3, the determination unit 712 determines that the boom operation unit 252b with the shortest position coordinate Pb among the multiple operation units 252a to 252r is the target operation unit because the distance from the position coordinate SP of the worker's hand U is less than or equal to a specific distance.

[0042] Then, in step S4, the information provision unit 713 displays some or all of the operation content information Ib1 to Ib3 (Figures 12 to 14) indicating the operation content of the boom operation unit 252b (target operation unit) on the display unit 51. Specifically, as shown in Figure 12, operation content information Ib1 is information that describes the operating state of the boom 22, which is the target of the operation, in accordance with boom swing operation (an operation to rotate the boom 22 around the pivot axis 221) and boom lift operation (an operation to raise and lower the boom 22 around the luffing axis 222) to the boom operation unit 252b, using images and text. Furthermore, as shown in Figure 13, operation content information Ib2 is information that describes the state of the boom swing operation to the boom operation unit 252b and the operating state of the boom 22, which is the target of the operation, in accordance with the state of the boom swing operation, using images. In addition, as shown in Figure 14, operation content information Ib3 is information that describes the state of the boom lift operation to the boom operation unit 252b and the operating state of the boom 22, which is the target of the operation, in accordance with the state of the boom lift operation, using images.

[0043] Here, the information providing unit 713 sequentially switches the operation content information Ib1 to Ib3 displayed on the display unit 51 depending on whether the user presses the "next" or "back" icons IC1 and IC2 (Figures 12 to 14), which correspond to the input device 6 composed of a touch panel, or depending on the elapsed time. In addition, the information providing unit 713 terminates the display of operation content information Ib1 to Ib3 on the display unit 51 in response to the user pressing the "end" icon IC3 (Figures 12 to 14), which corresponds to the input device 6 composed of a touch panel. Furthermore, the configuration is not limited to sequentially switching between displaying the operation information Ib1 to Ib3 on the display unit 51; it is also acceptable to configure the system to display all of the operation information Ib1 to Ib3 on the display unit 51.

[0044] According to the embodiment described above, the following effects are achieved. The information providing device 7 according to this embodiment includes a determination unit 712 that determines which of the multiple operation units 252a to 252r the operator intends to operate as the target operation unit, and an information providing unit 713 that provides operation content information indicating the operation content of the target operation unit. This operation content information is then provided to the operator via the notification device 5. Therefore, the information providing device 7 according to this embodiment can provide the worker with operation content information indicating the operation content of the control unit that the worker is trying to operate, thereby improving convenience.

[0045] In particular, the determination unit 712 determines that, among the multiple operating units 252a to 252r, the operating unit whose positional relationship with the position of the operator's hand U is a specific positional relationship is the target operating unit. Here, the operating unit that has this specific positional relationship is the operating unit among the multiple operating units 252a to 252r whose positional coordinate SP is the shortest distance from the position of the operator's hand U and is below a specific threshold. Therefore, before the operator actually operates the control unit, the system can determine which control unit the operator intends to operate is the target control unit and provide the operator with information about the operation of that target control unit.

[0046] Furthermore, the determination unit 712 recognizes the position coordinates SP of the worker's hand U through image recognition. Therefore, the hardware configuration of the operation support device 3 does not become complicated, and the position coordinates SP of the operator's hand U can be recognized well. In particular, the determination unit 712 recognizes the position coordinates SP of the worker's hand U by so-called skeletal detection. Therefore, there is no need to perform the cumbersome task of attaching markers or the like to the worker's hand U, and the position coordinates SP of the worker's hand U can be recognized accurately.

[0047] (Other embodiments) While embodiments for carrying out the present invention have been described so far, the present invention should not be limited to the embodiments described above. In the embodiments described above, the "information providing device" according to the present invention was mounted on an actual drilling machine 1, but the invention is not limited to this. For example, the "information providing device" according to the present invention may be mounted on a simulation device for acquiring skills in operating the drilling machine 1 without using the actual drilling machine 1. The simulation device includes a pseudo-operating device that includes a plurality of pseudo-operating units that simulate the operation device 25 which includes a plurality of operating units 252a to 252r, a display device such as a head-mounted display that displays images, and a control device that displays a virtual model (e.g., a 3D model) of the drilling machine 1 on the display device. Here, the control device simulates the operation of a part of the virtual model of the drilling machine 1, such as a boom, that corresponds to the pseudo-operating unit, on the display screen of the display device, in response to user operation on the pseudo-operating unit. When the "information providing device" according to the present invention is mounted on the simulation device, the "information providing device" determines the pseudo-operating unit that the user intends to operate as the target operating unit and provides operation content information indicating the operation content of the target operating unit. In other words, the "operating unit" according to the present invention may include the pseudo-operating unit described above.

[0048] In the embodiment described above, the determination unit 712 determined, based on the captured image generated by the imaging device 4, which of the multiple operation units 252a to 252r the operator intended to operate was the target operation unit; however, it is not limited to this. For example, without providing the imaging device 4, the determination unit 712 may determine which of the multiple operation units 252a to 252r was actually operated by the operator as the target operation unit. Alternatively, for example, without providing the imaging device 4, the determination unit 712 may detect the operation unit that is being held by the operator among the multiple operation units 252a to 252r, and determine that the detected operation unit is the target operation unit.

[0049] In the embodiment described above, the position coordinates Pa to Pr of the multiple operating parts 252a to 252r, and the position coordinate SP of the operator's hand U were configured using two-dimensional coordinates in the horizontal plane, but the invention is not limited to this. For example, the imaging device 4 is configured with a stereo camera. Then, by simultaneously capturing images from different viewpoints using the stereo camera and using the relative displacement amount in the images of the same subject, a stereo measurement technique is used to calculate the three-dimensional position of the subject based on the principle of triangulation. The three-dimensional coordinates, including the vertical height, may be expressed as position coordinates Pa~Pr,SP. Alternatively, for example, the imaging device 4 may be configured using a distance image sensor such as a TOF (Time Of Flight) sensor. Furthermore, the three-dimensional coordinates including vertical height may be expressed as position coordinates Pa~Pr,SP.

[0050] In the embodiment described above, the determination unit 712 recognized the position coordinates SP of the worker's hand U by image recognition using a learning model, but it is not limited to this. For example, a marker or the like can be attached to the worker's hand U. The determination unit 712 can then be configured to detect the marker using image recognition and recognize the position coordinates SP of the worker's hand U. Alternatively, for example, a light-emitting unit that emits infrared rays may be attached to the worker's hand U. Furthermore, an infrared camera that detects these infrared rays may be provided in the operation support device 3. The determination unit 712 may then be configured to recognize the position coordinates SP of the worker's hand U based on the position of the light-emitting unit detected by the infrared camera. [Explanation of Symbols]

[0051] 1 Drilling machine 2. Drilling machine body 3 Operation support device 4. Imaging device 5. Notification device 6 Input devices 7 Information provision device 21. Running bogie 22 Boom 23 Rock-cutting unit 24 Cockpit 25 Operating device 51 Display section 52 Audio output section 71 Control Unit 72 Memory section 211 Trucks 221 Swivel axis 222 Relief axis 231 Guide Shell 232 Rock drilling machine 233 Tools 241 Driver's seat 251 Control Panel 252a~252r Operation section 252a Drilling operation part 252b Boom Operating Section 521 Speaker 711 Image acquisition unit 712 Judgment section 713 Information Provision Department 721 Related Information Database 722 Learning Model DB F0~F2 captured images Operation details for Ia1-Ia4 and Ib1-Ib3 IC1~IC3 Icons IF index finger Pa~Pr,SP Position coordinates U Worker's hand TH Thumb

Claims

1. A determination unit that determines which of the multiple operating parts used to operate the drilling machine the user intends to operate is the target operating part, The system includes an information providing unit that provides operation content information indicating the operation content of the target operation unit, The determination unit and the information provision unit perform processing before the user operates the operation unit. The operation details include information describing the operating state of the target object corresponding to the target operation unit, The determination unit, An information providing device that measures the position coordinates of the user's hand, analyzes the positional relationship between the position coordinates of the user's hand and the pre-set position coordinates of the plurality of operating units, and, based on the results of the analysis, determines that the operating unit among the plurality of operating units whose positional relationship with the user's hand is a specific positional relationship is the target operating unit.

2. The system further includes an image acquisition unit that acquires data of a captured image of a region including the aforementioned multiple operating units, The determination unit, The information providing device according to claim 1, wherein the device measures the position coordinates of the user's hand based on the captured image, analyzes the positional relationship between the position coordinates of the user's hand and the pre-set position coordinates of the plurality of operating units, and determines the operating unit that has the specific positional relationship based on the results of the analysis as the target operating unit.

3. The determination unit, The information providing device according to claim 2, which recognizes the position of the user's hand by detecting the skeletal structure of the user's hand based on the captured image.

4. The determination unit, The information providing device according to claim 2 or 3, wherein, among the plurality of operating units, the operating unit whose position coordinates in the specific positional relationship are the shortest and whose distance from the position coordinates of the user's hand is below a specific threshold is determined to be the target operating unit.

5. An information provision method performed by an information provision device, A determination step in which, among the multiple operating parts used to operate the drilling machine, the operating part that the user intends to operate is determined to be the target operating part, The step includes reading operation content information indicating the operation content of the target operation unit from the storage unit and providing it, The determination step and the information provision step are performed before the user operates the operation unit. The operation details include information describing the operating state of the target object corresponding to the target operation unit, The aforementioned determination step is, An information provision method that measures the position coordinates of the user's hand, analyzes the positional relationship between the position coordinates of the user's hand and the pre-set position coordinates of the plurality of operating units, and, based on the results of the analysis, determines that the operating unit among the plurality of operating units whose positional relationship with the user's hand is a specific positional relationship is the target operating unit.

6. A determination step in which, among the multiple operating parts used to operate the drilling machine, the operating part that the user intends to operate is determined to be the target operating part, The computer is instructed to perform an information provision step which involves reading operation content information indicating the operation content of the target operation unit from the storage unit and providing it, The determination step and the information provision step are performed before the user operates the operation unit. The operation details include information describing the operating state of the target object corresponding to the target operation unit, The aforementioned determination step is, An information provision program that measures the positional coordinates of the user's hand, analyzes the positional relationship between the positional coordinates of the user's hand and the pre-set positional coordinates of the plurality of operating units, and, based on the results of the analysis, determines that the operating unit among the plurality of operating units whose positional relationship with the user's hand is a specific positional relationship is the target operating unit.

Images