Unmanned drill jumbo interaction method, device and system and medium

Abstract

The invention discloses an unmanned drill jumbo interaction method, device and system and a medium. The method comprises the steps that a control interface is displayed; a job display area, a start-stop trigger control and an operation trigger control are arranged on the control interface, and a target three-dimensional model is displayed in the job display area; determining that the start-stop trigger control is triggered, and establishing control on the unmanned drill jumbo; obtaining a target action instruction generated when the operation triggering control is triggered; the target action instruction is transmitted to the unmanned drill jumbo, so that the unmanned drill jumbo executes a corresponding action; adjusting the action of the target three-dimensional model according to the target action instruction; and determining a vibration feedback mode according to the target action instruction, and controlling an external vibration unit to perform vibration feedback according to the vibration feedback mode. According to the invention, interaction on operation of the unmanned drill jumbo is provided from two dimensions of vision and touch, and the interactivity of a user to the unmanned drill jumbo is improved. The invention is mainly used in the technical field of engineering vehicles.

Description

Technical Field

[0001] The present invention relates to the technical field of engineering vehicles, and particularly to an interaction method, device, system and medium for an unmanned rock drilling jumbo. Background Art

[0002] An unmanned rock drilling jumbo (also known as a drilling jumbo) is a rock drilling device specifically designed for drill and blast construction in tunnels and underground projects. The unmanned rock drilling jumbo mainly consists of a rock drill, a drill boom (the supporting, positioning and propulsion mechanism of the rock drill), a steel structure frame, a traveling mechanism and other necessary auxiliary equipment. Among them, the drill boom is a key component of the unmanned rock drilling jumbo, which is responsible for supporting, positioning and propelling the rock drill for drilling operations. The working mechanism mainly consists of a thruster, a drill boom, a slewing mechanism, a translation mechanism, etc.

[0003] Existing unmanned rock drilling jumbos generally control the working mechanism through a remote controller. When the remote controller is used for control, there is a lack of interaction and recording of the operating state, and the overall operation interactivity, convenience and usability are not good. Therefore, how to improve the control interactivity of the unmanned rock drilling jumbo is a technical problem that urgently needs to be studied in the industry. Summary of the Invention

[0004] The present invention provides an interaction method, device, system and medium for an unmanned rock drilling jumbo to solve one or more technical problems existing in the prior art, and at least provide a beneficial choice or create conditions.

[0005] The present invention provides an interaction method for an unmanned rock drilling jumbo, including: displaying a control interface; wherein, an operation display area, a start / stop trigger control and an operation trigger control for the unmanned rock drilling jumbo are arranged on the control interface, a three-dimensional model of the unmanned rock drilling jumbo is displayed in the operation display area, and the three-dimensional model is denoted as a target three-dimensional model; If it is determined that the start / stop trigger control is triggered, then control over the unmanned rock drilling jumbo is established; After it is determined that control over the unmanned rock drilling jumbo is established, an action instruction generated when the operation trigger control is triggered is obtained, and the action instruction is denoted as a target action instruction; The target action instruction is transmitted to the unmanned rock drilling jumbo so that the unmanned rock drilling jumbo executes the corresponding action; the action of the target three-dimensional model is adjusted according to the target action instruction so that the target three-dimensional model completes the corresponding action change; A vibration feedback mode is determined according to the type of the target action instruction, and an external vibration unit is controlled to perform vibration feedback according to the vibration feedback mode.

[0006] Further, the interactive method for the unmanned rock drilling jumbo further includes: recording the action changes of the target 3D model in real time to form an operation log video, and storing the operation log video in a local device for subsequent calling and viewing.

[0007] Further, the interactive method for the unmanned rock drilling jumbo further includes: a viewing trigger control is provided on the control interface; If it is determined that the viewing trigger control is triggered, a path selection interface is displayed, and the operation log video in the local device is determined through the path selection interface. The operation log video is recorded as the target operation log video; a first playback window is generated, and the target operation log video is played in the first playback window.

[0008] Further, the interactive method for the unmanned rock drilling jumbo further includes: a video teaching trigger control is provided on the control interface; If it is determined that the video teaching trigger control is triggered, the control of the unmanned rock drilling jumbo is disconnected, and a first sub-interface is displayed; a plurality of learning trigger controls are provided on the first sub-interface; if it is determined that the learning trigger control is triggered, the corresponding teaching video is found, and the teaching video is recorded as the target teaching video; a second playback window is generated, and the target teaching video is played in the second playback window.

[0009] Further, the plurality of learning trigger controls include: a first learning trigger control, a second learning trigger control, a third learning trigger control, a fourth learning trigger control, and a fifth learning trigger control; When the first learning trigger control is triggered, a teaching video recording the operation method of blasting construction operations is found; when the second learning trigger control is triggered, a teaching video recording the operation method of radial bolt operations is found; when the third learning trigger control is triggered, a teaching video recording the operation method of advanced support operations is found; when the fourth learning trigger control is triggered, a teaching video recording the operation method of foot locking operations is found; when the fifth learning trigger control is triggered, a teaching video recording the operation method of cable bolt operations is found.

[0010] Further, the plurality of learning trigger controls include: a sixth learning trigger control, a seventh learning trigger control, and an eighth learning trigger control; When the sixth learning trigger control is triggered, a teaching video recording the operation method of perimeter hole operations is found; when the seventh learning trigger control is triggered, a teaching video recording the operation method of relief hole operations is found; when the eighth learning trigger control is triggered, a teaching video recording the operation method of cut hole operations is found.

[0011] On the other hand, an interactive device for an unmanned rock drilling jumbo is provided, including: a processor and a memory, where the memory is used to store a computer-readable program; when the computer-readable program is executed by the processor, the processor is caused to implement the unmanned rock drilling jumbo interaction method as described in any one of the above technical solutions.

[0012] On the other hand, an interactive system for an unmanned rock drilling jumbo is provided, including: a display module, a establishment module, an acquisition module, an adjustment module, and a feedback module; The display module is used for: displaying a control interface; wherein, an operation display area, a start / stop trigger control, and an operation trigger control for the unmanned rock drilling jumbo are arranged on the control interface, and a three-dimensional model of the unmanned rock drilling jumbo is displayed in the operation display area, and the three-dimensional model is denoted as a target three-dimensional model; The establishment module is used for: determining that the start / stop trigger control is triggered, and then establishing control over the unmanned rock drilling jumbo; The acquisition module is used for: after determining that control over the unmanned rock drilling jumbo is established, acquiring an action instruction generated when the operation trigger control is triggered, and denoting the action instruction as a target action instruction; The adjustment module is used for: transmitting the target action instruction to the unmanned rock drilling jumbo so that the unmanned rock drilling jumbo executes a corresponding action; adjusting the action of the target three-dimensional model according to the target action instruction so that the target three-dimensional model completes a corresponding action change; The feedback module is used for: determining a vibration feedback mode according to the type of the target action instruction, and controlling an external vibration unit to perform vibration feedback according to the vibration feedback mode.

[0013] Furthermore, the interactive system for the unmanned rock drilling jumbo further includes: a recording module; the recording module is used for: recording the action change of the target three-dimensional model in real time to form an operation log video, and storing the operation log video in a local device for subsequent calling and viewing.

[0014] On the other hand, a computer-readable storage medium is provided, characterized in that a program executable by a processor is stored therein, and when the program executable by the processor is executed by the processor, it is used to implement the unmanned rock drilling jumbo interaction method as described in any one of the above technical solutions.

[0015] The present invention has at least the following beneficial effects: By setting up a control interface, the method of the present invention uses the operation display area 103 to display the target three-dimensional model, and realizes the control of the unmanned rock drilling jumbo by the user through operating the trigger control. At the same time, the change of the target three-dimensional model is used to visually reflect the action change of the unmanned rock drilling jumbo. In the way of vibration feedback, the control change of the unmanned rock drilling jumbo is reflected tactually. The present invention provides the interaction of the operation of the unmanned rock drilling jumbo from two dimensions of vision and touch, and improves the interactivity of the user with the unmanned rock drilling jumbo. The present invention also provides corresponding devices, systems and media. The beneficial effects of the devices, systems and media are similar to those of the method, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings are used to provide a further understanding of the technical solutions of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation to the technical solutions of the present invention.

[0017] Figure 1 is a flowchart of the steps of the interaction method of the unmanned rock drilling jumbo; Figure 2 is a schematic diagram of the device structure of the interaction device of the unmanned rock drilling jumbo; Figure 3 is a schematic diagram of the system connection structure of the interaction system of the unmanned rock drilling jumbo; Figure 4 is a schematic diagram of the interface structure of the control interface; Figure 5 is a schematic diagram of the interface structure of the first sub-interface; Figure 6 is a schematic diagram of the structure of the unmanned rock drilling jumbo. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0019] It should be noted that although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence.

[0020] For the convenience of description, some terms in the text will be explained below.

[0021] Trigger control. A trigger control is a user interface element that can trigger preset operations or events when a user interacts with the interface (such as clicking, inputting, selecting, etc.) or when specific conditions are met (such as time arrival, data change, etc.).

[0022] Common types of trigger controls include: Button: A common trigger control used to execute user click operations; it can be a text button, an icon button, or a combined button; trigger operations usually include submitting a form, opening a new page, executing a specific function, etc. TextBox: A control for inputting text information; operations such as validation, searching, or others can be triggered by changes in the input content. CheckBox: A control that provides multiple options for users to select; corresponding operations can be triggered when a user selects or deselects an option. RadioButton: A control that provides a set of mutually exclusive options for users to select; when a user selects an option, other options are automatically deselected, and corresponding operations are triggered. ComboBox / DropDownList: A control that provides a drop-down selection function; corresponding operations are triggered when a user selects an option. Timer: A control used to trigger operations after a specific time interval. Commonly used in periodic tasks, animation effects, etc.

[0023] Human-machine interface. The human-machine interface (HMI) refers to the communication medium or means between a human and a computer system, and is a platform for two-way information exchange of various symbols and actions between a human and a computer.

[0024] Intelligent device. An intelligent device (intelligent device) refers to any device, instrument, or machine with computing and processing capabilities and communication capabilities.

[0025] Reference Figure 1 、 Figure 4 、 Figure 5 and Figure 6 , Figure 1 is a flowchart of the steps of the interaction method for an unmanned rock drilling jumbo. Figure 4 is a schematic diagram of the interface structure of the control interface. Figure 5 is a schematic diagram of the interface structure of the first sub-interface. Figure 6 is a schematic diagram of the structure of an unmanned rock drilling jumbo.

[0026] The present invention is mainly for facilitating effective interaction when a user controls an unmanned rock drilling jumbo, thereby enhancing the interactivity of the user during the control process of the unmanned rock drilling jumbo.

[0027] The interactive method of this unmanned rock drilling jumbo mainly operates through intelligent devices. When the intelligent device is operating, the steps it can execute include: Step 1, display the control interface.

[0028] The intelligent device displays the control interface in its human-machine interaction interface. The function of the control interface is to provide an interface for human-machine interaction, so that users can control the unmanned rock drilling jumbo through the control interface. At the same time, set feedback can also be provided through the control interface.

[0029] Combined with Figure 4 , for this reason, a variety of controls are set on the control interface. Among them, an operation display area 103, a start-stop trigger control 204, and an operation trigger control for the unmanned rock drilling jumbo are set on the control interface.

[0030] The function of the operation display area 103 is to display the form of the unmanned rock drilling jumbo during the operation process. Therefore, a three-dimensional model of the unmanned rock drilling jumbo is displayed in the operation display area 103. For the convenience of description, the three-dimensional model is denoted as the target three-dimensional model. The target three-dimensional model restores the current state of the entire unmanned rock drilling jumbo in a virtual form according to a certain proportion.

[0031] The function of the operation trigger control is to provide virtual control buttons for users, facilitating users to control the unmanned rock drilling jumbo through the virtual control buttons.

[0032] Among them, in some further specific embodiments, the operation trigger control includes: a first omnidirectional joystick control 101, a second omnidirectional joystick control 102, a chassis mechanism control trigger 201, a boom mechanism control trigger 202, and a drilling mechanism control trigger 203. The first omnidirectional joystick control 101 provides controls in all directions by triggering according to the selected mechanism to be controlled. The second omnidirectional joystick control 102 provides controls in all directions by triggering according to the selected mechanism to be controlled. When the user needs to control the unmanned rock drilling jumbo, they can trigger the start-stop trigger control 204.

[0033] Combined with Figure 6 , among them, in some further specific embodiments, the structure of the unmanned rock drilling jumbo includes: a chassis mechanism 901, a boom mechanism 902, a drilling mechanism 903, a visual communication system 904, a cockpit system 905, a power system 906, etc. The visual communication system 904 includes: millimeter-wave radar, lidar, cameras, perception cameras, integrated navigation, and 5G CPE, etc.

[0034] Due to the harsh working environment of the unmanned rock drilling jumbo, with large variations in temperature and light intensity, and it is often located in places with weak signal reception such as mine tunnels and tunnels, environmental perception is provided for the vehicle by installing millimeter-wave radar, lidar, monocular camera, perception camera, and signal antenna group on the rock drilling jumbo.

[0035] The unmanned operation of the rock drilling jumbo is controlled by a remote driving control center console, and the operator controls it on the remote console. During the unmanned operation of the rock drilling jumbo, the CAN data exchange storage on the rock drilling jumbo will store data on the actions of each actuator and the vehicle state on the rock drilling jumbo in real time. This data will be recorded in the unmanned driving system database and transmitted back to the remote service control center. At the same time, different connection methods and combinations of working tools of the rock drilling jumbo are selected for different working scenarios. Inside the tunnel, a local area network connection method is adopted, and a base station is established outside the tunnel to perform unmanned driving on the rock drilling jumbo within a certain distance. In the usage scenario where the 5G signal reception is normal, a 5G network connection is adopted, which is not restricted by distance. Detect the time delay and reliability of data reception and acquisition of the unmanned driving system under different connection methods, and record the manual control data to provide data comparison for the next stage. The unmanned driving of the rock drilling jumbo can achieve semi-automatic driving. After being corrected by the data collected in the previous stage, a database is formed and will be recorded in the unmanned intelligent driving control center of the rock drilling jumbo. At this time, the operation of the rock drilling jumbo will rely on the data provided by each sensing element to the unmanned driving control center for action analysis and determination of the working target. The CAN data exchange storage will still be an important basis for judging the working state of the vehicle. The operator is responsible for monitoring at this time, and the information on the rock drilling jumbo will also be monitored in real time to ensure the normal operation of the rock drilling jumbo. Detect whether the information collected in the data acquisition stage of the unmanned driving system is reliable, conduct experiments on it, and record the information during its operation process. The unmanned intelligent driving control center will merge data with the entire vehicle controllable center of the rock drilling jumbo to form a whole. The rock drilling jumbo will perform fully automatic driving to complete the positioning of blast holes, anchor holes, and grouting holes, and can achieve functions such as drilling, feedback, and adjustment. Due to different working environments, there will be certain limitations for sensors. The combination of millimeter-wave radar and lidar adopted on the rock drilling jumbo can effectively avoid the perception obstacles caused by environmental problems. At the same time, the monocular camera and perception camera provide effective environmental perception for the unmanned driving center. The remote control center console will still act as a monitor, and manual intervention will only occur when data calculation errors or abnormal operation of the rock drilling jumbo occur. Maximize its degree of automation and provide data analysis for later improvement.

[0036] Step 2: When it is determined that the start / stop trigger control 204 is triggered, control of the unmanned rock drilling jumbo is established.

[0037] After the intelligent device determines that the start-stop trigger control 204 is triggered, it can be considered that the user needs to control the unmanned rock drilling jumbo. Therefore, it will establish a communication link with the unmanned rock drilling jumbo to take over the control of the unmanned rock drilling jumbo.

[0038] Step 3: After determining that the control of the unmanned rock drilling jumbo is established, obtain the action instruction generated when the operation trigger control is triggered, and record the action instruction as the target action instruction.

[0039] When the user triggers the operation trigger control, the operation trigger control will generate a corresponding action instruction according to the trigger situation. After the intelligent device determines that the communication with the unmanned rock drilling jumbo has been established and control can be performed, the intelligent device will obtain the action instruction. For the convenience of description, the action instruction is recorded as the target action instruction.

[0040] Step 4: Transmit the target action instruction to the unmanned rock drilling jumbo so that the unmanned rock drilling jumbo executes the corresponding action; adjust the action of the target 3D model according to the target action instruction so that the target 3D model completes the corresponding action change.

[0041] After obtaining the target action instruction, the intelligent device can transmit the target action instruction to the unmanned rock drilling jumbo, so that the unmanned rock drilling jumbo can execute the target action instruction and complete the corresponding action. For example, the forward or backward movement of the unmanned rock drilling jumbo. The intelligent device will also adjust the target 3D model according to the target action instruction, so that the target 3D model forms a corresponding action change relative to the unmanned rock drilling jumbo. Thus, the target 3D model can reflect the action of the unmanned rock drilling jumbo.

[0042] Step 5: Determine the vibration feedback mode according to the type of the target action instruction, and control the external vibration unit to perform vibration feedback according to the vibration feedback mode.

[0043] After the intelligent device determines the target action instruction, it will determine the vibration feedback mode through the type of the target action instruction. For example, when it is determined that the target action instruction is the forward movement type of the unmanned rock drilling jumbo, then at this time it is the vibration feedback mode with the first frequency and the first vibration intensity. The intelligent device will control the external vibration unit according to this vibration feedback mode, so that the vibration unit generates vibration to feedback the target action instruction to remind the user.

[0044] The present invention provides a control interface, uses the operation display area 103 to display the target three-dimensional model, and enables the user to control the unmanned rock drilling jumbo by operating the trigger control. Meanwhile, the change of the target three-dimensional model is used to visually reflect the action change of the unmanned rock drilling jumbo. In a way of vibration feedback, the control change of the unmanned rock drilling jumbo is reflected tactually. The present invention provides interaction in the operation of the unmanned rock drilling jumbo from two dimensions of vision and touch, improving the interactivity of the user with the unmanned rock drilling jumbo.

[0045] To facilitate the user to view the operation of the current unmanned rock drilling jumbo afterwards, in some further specific embodiments, the intelligent device will record the action change of the target three-dimensional model in real time, and form an operation log video through the real-time record. To facilitate the user to view the operation log video, the intelligent device will store the operation log video in the local device. The user can view the operation log video through the local device, and thus view the corresponding operation process through the operation log video.

[0046] To enable the user to view the corresponding operation log video through the control interface, in some further specific embodiments, the control interface is further provided with a viewing trigger control. When the user needs to view the operation log video, he can trigger the viewing trigger control. After the intelligent device determines that the viewing trigger control is triggered, it will display a path selection interface. The user determines the operation log video to be viewed from the local device through the trigger operation of the path selection interface. For the convenience of description, the determined operation log video is denoted as the target operation log video. The intelligent device will generate a first playback window and play the target operation log video in the first playback window.

[0047] The user can view the target operation log video through the control interface, improving the usability of the user to view the operation log video.

[0048] When a novice user operates, there may be some problems in the operation. In the prior art, when encountering these problems, the instruction manual of the unmanned rock drilling jumbo is generally queried to master how to operate. This method is very troublesome and the learning effect is not good. To enable the user to learn more conveniently, a video teaching trigger control 205 is further provided on the control interface. By setting the video teaching trigger control 205 on the control interface, the user can timely and conveniently trigger the video teaching trigger control 205 when encountering operation difficulties to find the desired teaching video.

[0049] When the user needs to conduct temporary learning, they can trigger the video teaching trigger control 205. After the intelligent device determines that the video teaching trigger control 205 has been triggered, it will disconnect the control of the unmanned rock drilling jumbo. By disconnecting the control of the unmanned rock drilling jumbo, it is possible to avoid accidental triggering of the control of the unmanned rock drilling jumbo during the learning process. Then, the intelligent device will display the first sub-interface.

[0050] A number of learning trigger controls are set on the first sub-interface, and each learning trigger control corresponds to a corresponding teaching video.

[0051] When the user needs to view a certain teaching video, they can trigger the corresponding learning trigger control. After the intelligent device determines that the user has triggered the learning trigger control, it will find the corresponding teaching video. For the sake of convenience of description, the teaching video is denoted as the target teaching video. The intelligent device will generate a second playback window and play the target teaching video through the second playback window. Thus, the user can view the target teaching video through the second playback window, achieving the purpose of learning the relevant operation methods.

[0052] Combined Figure 5 In some further specific embodiments, the number of learning trigger controls includes: a first learning trigger control 301, a second learning trigger control 302, a third learning trigger control 303, a fourth learning trigger control 304, a fifth learning trigger control 305, a sixth learning trigger control 306, a seventh learning trigger control 307, and an eighth learning trigger control 308.

[0053] When the intelligent device determines that the user has triggered the first learning trigger control 301, it will find the teaching video recording the operation method of the blasting construction operation. Thus, it is a teaching video guiding the user on the necessary operations of the unmanned rock drilling jumbo in the blasting construction operation.

[0054] When the intelligent device determines that the user has triggered the second learning trigger control 302, it will find the teaching video recording the operation method of the radial bolt operation. Thus, it is a teaching video guiding the user on the necessary operations of the unmanned rock drilling jumbo in the radial bolt operation.

[0055] When the intelligent device determines that the user has triggered the third learning trigger control 303, it will find the teaching video recording the operation method of the advanced support operation. Thus, it is a teaching video guiding the user on the necessary operations of the unmanned rock drilling jumbo in the advanced support operation.

[0056] When the intelligent device determines that the user triggers the fourth learning trigger control 304, it finds the teaching video recording the operation method of the leg-locking operation. Thus, it guides the user on how to perform the necessary operations regarding the unmanned rock drilling jumbo in the leg-locking operation.

[0057] When the intelligent device determines that the user triggers the fifth learning trigger control 305, it finds the teaching video recording the operation method of the cable bolt operation. Thus, it guides the user on how to perform the necessary operations regarding the unmanned rock drilling jumbo in the cable bolt operation.

[0058] When the intelligent device determines that the user triggers the sixth learning trigger control 306, it finds the teaching video recording the operation method of the perimeter hole operation. Thus, it guides the user on how to perform the necessary operations regarding the unmanned rock drilling jumbo in the perimeter hole operation.

[0059] When the intelligent device determines that the user triggers the seventh learning trigger control 307, it finds the teaching video recording the operation method of the relief hole operation. Thus, it guides the user on how to perform the necessary operations regarding the unmanned rock drilling jumbo in the relief hole operation.

[0060] When the intelligent device determines that the user triggers the eighth learning trigger control 308, it finds the teaching video recording the operation method of the cut hole operation. Thus, it guides the user on how to perform the necessary operations regarding the unmanned rock drilling jumbo in the cut hole operation.

[0061] On the other hand, referring to Figure 2 , Figure 2 is the schematic diagram of the device structure of the unmanned rock drilling jumbo interaction device.

[0062] Provided is an unmanned rock drilling jumbo interaction device, including: a processor and a memory; wherein, the memory is used for storing a computer-readable program. When the computer-readable program is executed by the processor, the processor implements the unmanned rock drilling jumbo interaction method described in any one of the above technical solutions.

[0063] Those of ordinary skill in the art can understand that all or some of the steps and systems disclosed in the above methods can be implemented as software, firmware, hardware, and their appropriate combinations. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. It is well known to those of ordinary skill in the art that communication media typically contains computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0064] On the other hand, referring to Figure 3 , Figure 3 is a schematic diagram of the system connection structure of an unmanned rock drilling jumbo interaction system.

[0065] An unmanned rock drilling jumbo interaction system is provided, including: a display module, a establishment module, an acquisition module, an adjustment module, and a feedback module.

[0066] The display module is used to: display a control interface.

[0067] The display module displays the control interface through its human-machine interaction interface. The function of the control interface is to provide an interface for human-machine interaction, so that the user can control the unmanned rock drilling jumbo through the control interface. At the same time, set feedback can also be provided through the control interface.

[0068] For this reason, a variety of controls will be set on the control interface. Among them, an operation display area 103 and an operation trigger control for the unmanned rock drilling jumbo are set on the control interface.

[0069] The function of the operation display area 103 is to display the form of the unmanned rock drilling jumbo during the operation process. Therefore, a three-dimensional model of the unmanned rock drilling jumbo is displayed in the operation display area 103. For the convenience of description, the three-dimensional model is denoted as the target three-dimensional model. The target three-dimensional model restores the current state of the entire unmanned rock drilling jumbo in a virtual form according to a certain proportion.

[0070] The function of the operation trigger control is to provide virtual control buttons for the user, facilitating the user to control the unmanned rock drilling jumbo through the virtual control buttons.

[0071] Among them, in some further specific embodiments, the operation trigger control includes: a first omnidirectional joystick control 101, a second omnidirectional joystick control 102, a chassis mechanism control trigger 201, a boom mechanism control trigger 202, and a drilling mechanism control trigger 203. The first omnidirectional joystick control 101 provides controls in all directions by triggering according to the selected mechanism to be controlled. The second omnidirectional joystick control 102 provides controls in all directions by triggering according to the selected mechanism to be controlled.

[0072] When the user needs to control the unmanned rock drilling jumbo, they can trigger the start-stop trigger control 204.

[0073] The establishment module is used to: determine that the start-stop trigger control 204 is triggered, and then establish the control of the unmanned rock drilling jumbo.

[0074] After the establishment module determines that the start-stop trigger control 204 is triggered, it can be considered that the user needs to control the unmanned rock drilling jumbo. Therefore, it will establish a communication link with the unmanned rock drilling jumbo to take over the control of the unmanned rock drilling jumbo.

[0075] The acquisition module is used to: after determining that the control of the unmanned rock drilling jumbo is established, acquire the action instruction generated when the operation trigger control is triggered, and denote the action instruction as the target action instruction.

[0076] When the user triggers the operation trigger control, the operation trigger control will generate a corresponding action instruction according to the triggering situation. After the acquisition module determines that the communication with the unmanned rock drilling jumbo has been established and control can be performed, the acquisition module will acquire the action instruction. For the convenience of description, the action instruction is denoted as the target action instruction.

[0077] The adjustment module is used to: transmit the target action instruction to the unmanned rock drilling jumbo so that the unmanned rock drilling jumbo executes the corresponding action; adjust the action of the target three-dimensional model according to the target action instruction so that the target three-dimensional model completes the corresponding action change.

[0078] After obtaining the target action instruction, the adjustment module can transmit the target action instruction to the unmanned rock drilling jumbo, so that the unmanned rock drilling jumbo can execute the target action instruction to complete the corresponding action. For example, the forward or backward movement of the unmanned rock drilling jumbo. The adjustment module will also adjust the target 3D model according to the target action instruction, so that the target 3D model forms an action change corresponding to the unmanned rock drilling jumbo. Thus, the target 3D model can reflect the action of the unmanned rock drilling jumbo.

[0079] The feedback module is used to: determine the vibration feedback mode according to the type of the target action instruction, and control the external vibration unit to perform vibration feedback according to the vibration feedback mode.

[0080] After determining the target action instruction, the feedback module will determine the vibration feedback mode through the type of the target action instruction. For example, when it is determined that the target action instruction is the forward movement type of the unmanned rock drilling jumbo, then at this time it is the vibration feedback mode with the first frequency and the first vibration intensity. The feedback module will control the external vibration unit according to this vibration feedback mode, so that the vibration unit generates vibration to feedback the target action instruction to remind the user.

[0081] In order to facilitate the user to view the operation of the current unmanned rock drilling jumbo afterwards, in some further specific embodiments, the unmanned rock drilling jumbo interaction system further includes a recording module.

[0082] The recording module will record the action changes of the target 3D model in real time, and form an operation log video through the real-time recording. In order to facilitate the user to view the operation log video, the recording module will store the operation log video in the local device. The user can view the operation log video through the local device, so as to view the corresponding operation process through the operation log video.

[0083] On the other hand, a computer-readable storage medium is provided, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the unmanned rock drilling jumbo interaction method described in any one of the above specific embodiments.

[0084] The embodiment of the present application also discloses a computer program product, including a computer program or computer instruction. The computer program or computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer program or computer instruction from the computer-readable storage medium, and the processor executes the computer program or computer instruction, so that the computer device executes the unmanned rock drilling jumbo interaction method described in any of the previous embodiments.

[0085] In the description of this application and the above-mentioned drawings, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0086] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist simultaneously. Here, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or a similar expression means any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0087] In several embodiments provided by this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be an indirect coupling or communication connection through some interfaces, devices, or units, and can be in electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0090] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0091] Although the description of the present application has been quite detailed and several of the described embodiments have been described in particular, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be regarded as effectively covering the intended scope of the present application by referring to the appended claims and considering the broad possibilities of interpretation of these claims in light of the prior art. In addition, the present application has been described above with embodiments foreseeable by the inventors for the purpose of providing a useful description, and non-substantive modifications to the present application that are not currently foreseeable can still represent equivalent modifications of the present application.

Claims

1. An unmanned drilling rig interaction method, characterized in that: include: A display control interface; wherein the control interface is provided with an operation display area, a start / stop trigger control, and an operation trigger control of the unmanned drilling rig, a three-dimensional model of the unmanned drilling rig is displayed in the operation display area, and the three-dimensional model is recorded as a target three-dimensional model; Determining that the start-stop trigger control is triggered, then establishing control over the unmanned drilling rig; After determining that control is established with the unmanned drilling rig, obtaining an action instruction generated when the operation trigger control is triggered, and recording the action instruction as a target action instruction; The target action instruction is transmitted to the unmanned rock drilling rig so that the unmanned rock drilling rig performs a corresponding action; the action of the target three-dimensional model is adjusted according to the target action instruction so that the target three-dimensional model completes the corresponding action change; A vibration feedback mode is determined according to the type of the target action instruction, and an external vibration unit is controlled to perform vibration feedback according to the vibration feedback mode.

2. The unmanned drilling rig interaction method according to claim 1, characterized in that: Also includes: The action changes of the target three-dimensional model are recorded in real time to form an operation log video, and the operation log video is stored in a local device for future viewing.

3. The unmanned drilling rig interaction method according to claim 1, characterized in that: Also includes: The control interface is provided with a viewing trigger control; If it is determined that the viewing trigger control is triggered, a path selection interface is displayed, and an operation log video in a local device is determined through the path selection interface, and the operation log video is recorded as a target operation log video; A first play window is generated, and the target operation log video is played in the first play window.

4. The unmanned drilling rig interaction method according to claim 1, characterized in that: Also includes: The control interface is provided with a video teaching trigger control; If it is determined that the video teaching trigger control is triggered, the control of the unmanned rock drilling rig is disconnected and the first sub-interface is displayed; the first sub-interface is provided with a plurality of learning trigger controls; if it is determined that the learning trigger control is triggered, the corresponding teaching video is found and the teaching video is recorded as the target teaching video; Generate a second play window, and play the target teaching video in the second play window.

5. The unmanned drilling rig interaction method according to claim 4, characterized in that: The plurality of learning trigger controls include: a first learning trigger control, a second learning trigger control, a third learning trigger control, a fourth learning trigger control, and a fifth learning trigger control; When the first learning trigger control is triggered, a teaching video recording the operating method of blasting construction operations is found; when the second learning trigger control is triggered, a teaching video recording the operating method of radial anchor operations is found; when the third learning trigger control is triggered, a teaching video recording the operating method of advance support operations is found; when the fourth learning trigger control is triggered, a teaching video recording the operating method of locking foot operations is found; when the fifth learning trigger control is triggered, a teaching video recording the operating method of cable anchor operations is found.

6. The unmanned drilling rig interaction method according to claim 4, characterized in that: The plurality of learning trigger controls include: a sixth learning trigger control, a seventh learning trigger control, and an eighth learning trigger control; When the sixth learning trigger control is triggered, a teaching video recording the operating method of peripheral eye operations is found; when the seventh learning trigger control is triggered, a teaching video recording the operating method of auxiliary eye operations is found; when the eighth learning trigger control is triggered, a teaching video recording the operating method of groove eye operations is found.

7. An unmanned drilling trolley interactive device, characterized in that: include: processor; A memory for storing a computer readable program; When the computer-readable program is executed by the processor, the processor implements the unmanned drilling rig interaction method according to any one of claims 1 to 6.

8. An unmanned drilling rig interactive system, characterized in that: include: Display module, build module, acquire module, adjust module and feedback module; The display module is used to: display a control interface; wherein the control interface is provided with an operation display area, a start / stop trigger control, and an operation trigger control of the unmanned drilling rig; a three-dimensional model of the unmanned drilling rig is displayed in the operation display area, and the three-dimensional model is recorded as a target three-dimensional model; The establishment module is used to: determine that the start-stop trigger control is triggered, and then establish control over the unmanned rock drilling rig; The acquisition module is used to: after determining that control is established with the unmanned drilling rig, acquire the action instruction generated when the operation trigger control is triggered, and record the action instruction as the target action instruction; The adjustment module is used to: transmit the target action instruction to the unmanned rock drilling rig so that the unmanned rock drilling rig performs a corresponding action; adjust the action of the target three-dimensional model according to the target action instruction so that the target three-dimensional model completes the corresponding action change; The feedback module is used to determine a vibration feedback mode according to the type of the target action instruction, and control an external vibration unit to perform vibration feedback according to the vibration feedback mode.

9. The unmanned drilling rig interactive system according to claim 8, characterized in that: Also includes: Recording module; The recording module is used to record the action changes of the target three-dimensional model in real time to form an operation log video, and store the operation log video in a local device for future viewing.

10. A computer-readable storage medium, characterized in that: A processor-executable program is stored therein, and when the processor-executable program is executed by the processor, it is used to implement the unmanned drilling rig interaction method as described in any one of claims 1 to 6.

Images