Remote control system for engineering machinery

Abstract

The embodiment of the utility model provides an engineering machinery remote control system, and belongs to the technical field of engineering machinery. The engineering machinery remote control system comprises a first display screen and a remote control handle which are arranged at a remote control end; the image processing host, the wireless communication module, the laser radar device and the plurality of camera devices are arranged on the engineering machinery; the remote control handle is used for remotely controlling the engineering machinery, signal output ends of the laser radar device and the plurality of camera devices are respectively connected with a signal input end of the image processing host, and a signal output end of the image processing host is connected with a signal input end of the first display screen through the wireless communication module. The short-range remote control of the engineering machinery with video monitoring and laser radar monitoring is realized, and video information and three-dimensional size and position distance information of an object can be provided for an operator, so that the efficiency and the safety of remotely controlling the engineering machinery are improved.

Description

Technical Field

[0001] The utility model relates to the technical field of engineering machinery, in particular to an engineering machinery remote control system. Background Art

[0002] With the development of science and technology, construction machinery is being used more and more widely and is becoming more and more intelligent. For example, excavators are one of the important tools for modern engineering construction and production. They are widely used in earthwork infrastructure, water conservancy and transportation, mining and excavation and other fields. Their main functions include excavation, loading, transportation and leveling.

[0003] However, in some work situations with potential safety hazards, such as cleaning high-temperature boiler slag, or in work environments prone to landslides or collapses, operators need to take risks to work. Existing short-range remote control systems can only provide video images, but cannot provide operators with depth information of objects. Operators cannot perceive the three-dimensional size and position distance of objects through their eyes like in the cab, which greatly reduces the operator's work efficiency. Utility Model Content

[0004] The purpose of the embodiment of the utility model is to provide a remote control system for engineering machinery, which can provide the operator with video information and three-dimensional size and position distance information of an object, thereby helping to improve the efficiency of remote control engineering machinery.

[0005] In order to achieve the above-mentioned purpose, the embodiment of the utility model provides a remote control system for engineering machinery, comprising:

[0006] A first display screen and a remote control handle provided at the remote control end;

[0007] An image processing host, a wireless communication module, a laser radar device and a plurality of camera devices installed on the engineering machinery;

[0008] The remote control handle is used to remotely control the engineering machinery. The signal output ends of the laser radar equipment and multiple camera equipment are respectively connected to the signal input ends of the image processing host. The signal output end of the image processing host is connected to the signal input end of the first display screen through the wireless communication module.

[0009] In an embodiment of the present application, the remote control end is further provided with a fixed bracket, and the display screen is mounted on the remote control handle through the fixed bracket.

[0010] In the embodiment of the present application, the angle of the fixing bracket is adjustable.

[0011] In the embodiment of the present application, a shoulder strap is provided on the remote control handle.

[0012] In the embodiment of the present application, a controller is provided on the engineering machinery, and the controller transmits data with the remote control handle through the wireless communication module.

[0013] In an embodiment of the present application, the wireless communication module includes a wireless transmission module and a remote control receiving module. The signal output end of the image processing host is connected to the signal input end of the wireless transmission module, and the signal output end of the wireless transmission module is connected to the signal input end of the first display screen; the controller is connected to the remote control receiving module, and the remote control receiving module is connected to the remote control handle.

[0014] In the embodiment of the present application, a second display is further provided on the engineering machinery, and a signal input end of the second display is connected to a signal output end of the controller.

[0015] In the embodiment of the present application, the signal output terminal of the image processing host is connected to the signal input terminal of the second display.

[0016] In the embodiment of the present application, a sun visor is provided on the first display screen.

[0017] In the embodiment of the present application, the signal output end of each camera device is connected to the signal input end of the image processing host through an AHD data line.

[0018] Through the above technical scheme, the laser radar device and multiple camera devices respectively send the field image data collected by each to the image processing host, and the image processing host sends the field image data to the first display screen for display through the wireless communication module. The operator can view the field image data on the first display screen, and then remotely control the construction machinery according to the field image data, thereby realizing short-range remote control of construction machinery with video monitoring and laser radar monitoring. Since the laser radar can sense the three-dimensional size and position distance of an object, it can provide the operator with video information and the three-dimensional size and position distance information of the object, which helps to improve the efficiency and safety of remote control construction machinery.

[0019] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the embodiments of the present utility model and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present utility model, but do not constitute a limitation on the embodiments of the present utility model. In the accompanying drawings:

[0021] Figure 1 The figure schematically shows a system topology diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The specific implementation of the embodiment of the utility model is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the utility model, and is not used to limit the embodiment of the utility model.

[0023] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0024] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0025] It should be noted that, in order to facilitate the description of the solution, this embodiment mainly describes the solution by taking the engineering machinery as an excavator.

[0026] Please see Figure 1 , Figure 1 Schematically shows a system topology diagram according to an embodiment of the present application. This embodiment provides a remote control system for construction machinery, including: a first display screen and a remote control handle arranged at a remote control end; an image processing host, a wireless communication module, a laser radar device and a plurality of camera devices arranged on the construction machinery; the remote control handle is used to remotely control the construction machinery, the signal output ends of the laser radar device and the plurality of camera devices are respectively connected to the signal input end of the image processing host, and the signal output end of the image processing host is connected to the signal input end of the first display screen through the wireless communication module.

[0027] In this embodiment, the laser radar device and multiple camera devices respectively send the field image data collected by each of them to the image processing host. After processing, the image processing host sends the image data to the first display screen for display through the wireless communication module.

[0028] The first display screen is arranged at the remote control end, so that the operator can view the live image at the remote control end. The first display screen can be a wireless display screen, which can be powered by an independent mobile power supply and used to display video surveillance, 540-degree panoramic images, and laser radar point cloud images at various azimuth angles. It can also have a menu selection function, and the display mode can be adjusted. The display mode can be selected to display one picture, two pictures, three pictures, four pictures, five pictures, six pictures, etc. Each display mode can be selected arbitrarily in multiple images (pictures of four cameras, 540-degree panoramic images, laser radar point cloud images). The wireless display screen can adjust the brightness, contrast, and color saturation of the displayed picture. The wireless display screen can also have a touch screen function, which can view the pixel coordinates and grayscale of any point in the video image, or the three-dimensional coordinates of any point in the laser radar point cloud map or the distance between any two points. The wireless display screen can also insert a memory card to save and replay the images and point cloud data machines received within a period of time.

[0029] In some embodiments, a sun visor is disposed on the first display screen.

[0030] In this embodiment, the first display screen can be equipped with a sun visor. In strong daylight, the sun visor can block the sunlight and prevent screen reflection. At the same time, it can increase the brightness. In the dim evening, the brightness can be reduced, thereby helping the operator to clearly obtain on-site image data on site.

[0031] The wireless communication module includes a wireless transmitter box and an antenna. The wireless transmitter box sends the received image signal, i.e., the on-site image data, to the wireless display screen through the antenna for display. The wireless communication module also includes a remote control receiver and a corresponding antenna for receiving signals. In order to facilitate the reception and transmission of signals, the antenna can be installed outside the engineering machinery, and the wireless transmitter box and the remote control receiver can be installed inside the engineering machinery. Taking the engineering machinery as an excavator as an example, the wireless transmitter box and the remote control receiver can be installed in the air filter compartment of the excavator, and the antenna can be installed above the cab.

[0032] The camera device may be a high-definition camera for collecting and shooting multiplied images and videos. In order to obtain more comprehensive on-site information, multiple camera devices are provided, and the number of camera devices may be determined specifically according to the construction machinery.

[0033] In some embodiments, the plurality of camera devices are respectively located around the engineering machinery, and a signal output end of each camera device is connected to a signal input end of the image processing host via an AHD data line.

[0034] In this embodiment, AHD (Analog High Definition Technology) can realize the reliable transmission of ultra-long distance (500 meters) high-definition video signals on the existing analog transmission line, can effectively reduce the color noise in the high-frequency area, and has better image restoration. It is connected to the image processing host through the AHD data line, which helps to obtain a clearer image. Taking the construction machinery as an excavator as an example, four high-definition cameras can be set up and installed around the excavator body.

[0035] By arranging video cameras around the construction machinery and connecting them to the image processing host via an AHD data cable, it is helpful to obtain clearer and more comprehensive on-site images, which helps the operator to accurately understand the on-site situation.

[0036] The above-mentioned laser radar device can be a solid-state laser radar or a mechanical laser radar. The on-site image data collected by the above-mentioned laser radar device is a point cloud image. The laser radar device is connected to the image processing host through a network cable. The laser radar device can be installed above the body of the engineering machinery. For example, if the engineering machinery is an excavator, the laser radar is installed horizontally above the excavator cab. The laser radar inputs the collected point cloud data to the image processing host through the network cable.

[0037] The above-mentioned image processing host is used to process image data. It can generate a 540-degree panoramic image after panoramic stitching of multiple on-site images collected by multiple camera devices, and then send the 540-degree panoramic image and the on-site image data collected by the laser radar device to the first display screen for display. The image processing host can be installed in the engineering machinery. Taking the engineering machinery as an excavator as an example, the image processing host is installed in the air filter compartment of the excavator. The video images of the four cameras are panoramically stitched to generate a 540-degree panoramic image. The video images of each camera and the stitched video images and point cloud images are transmitted to the wireless transmitter box through the AHD data line. The above-mentioned image processing host can also deploy a deep learning algorithm to detect the human body and its distance in the video or image. If the distance between the person and the engineering machinery is less than a certain threshold, an alarm will be issued to remind the operator to operate, which can improve the safety during the construction operation. The image processing host can also have a large-capacity storage function, which can save the video or image data collected by each camera for at least one consecutive day, as well as the point cloud data collected by the laser radar.

[0038] The remote control handle can be powered by an independent mobile power source, and can be equipped with an LCD display screen for displaying the status parameter information of the engineering machinery, and can also be equipped with switches such as rockers, knobs, and buttons for operating the remote control engineering machinery. An emergency stop button function is also provided to disconnect the remote control of the engineering machinery in an emergency in the event of an accident. The operator can operate the remote control handle, or observe the LCD display screen on the handle to obtain the status parameters of the excavator.

[0039] In the above implementation process, the laser radar device and multiple camera devices respectively send the field image data collected by each to the image processing host, and the image processing host sends the field image data to the first display screen for display through the wireless communication module. The operator can view the field image data on the first display screen, and then remotely control the engineering machinery according to the field image data, realizing short-range remote control of engineering machinery with video monitoring and laser radar monitoring. Since the laser radar can perceive the three-dimensional size and position distance of the object, it can provide the operator with video information and the three-dimensional size and position distance information of the object, which helps to improve the efficiency and safety of remote control engineering machinery. The above image processing host can also generate a 540-degree panoramic image after panoramic stitching of multiple field images collected by multiple camera devices, so that the short-range remote control of engineering machinery with 540-degree panoramic video monitoring and laser radar monitoring can be achieved, further improving the efficiency and safety of remote control engineering machinery. The above image processing host also has a deep learning video detection function. When it detects that the distance between a person and the excavator is less than a certain threshold, a warning will be issued, which can improve the safety during the construction operation.

[0040] In some embodiments, the remote control end is further provided with a fixing bracket, and the display screen is mounted on the remote control handle through the fixing bracket.

[0041] In this embodiment, the fixing bracket is used to fix the first display screen to the remote control handle, so that it is convenient to view the information on the first display screen when remotely controlling the engineering machinery, thereby further improving the efficiency of remotely controlling the engineering machinery.

[0042] In some embodiments, the angle of the fixing bracket is adjustable.

[0043] In this embodiment, the fixed bracket has an angle adjustment function, which is convenient for the operator to view the video or point cloud map at any angle.

[0044] In some embodiments, a shoulder strap is provided on the remote control handle.

[0045] In this embodiment, the above-mentioned shoulder strap includes a shoulder strap and a waist belt, which are used to wear and fix the remote control handle on the operator to facilitate the operator to perform remote control.

[0046] In some embodiments, a controller is provided on the engineering machine, and the controller communicates with the remote control handle through the wireless communication module.

[0047] In this embodiment, the controller is used to control the engineering machinery. The remote control handle sends instructions to the controller, and the controller controls the engineering machinery to work according to the instructions, thereby realizing the remote control function.

[0048] In some embodiments, the wireless communication module includes a wireless transmission module and a remote control receiving module, the signal output end of the image processing host is connected to the signal input end of the wireless transmission module, and the signal output end of the wireless transmission module is connected to the signal input end of the first display screen to send the on-site image data to the first display screen for display; the controller is connected to the remote control receiving module, and the remote control receiving module is connected to the remote control handle.

[0049] In this embodiment, the wireless transmission module includes a wireless transmission box and an antenna. The image processing host transmits the video image signal collected by the camera and the generated panoramic image and the laser radar point cloud map to the wireless transmission box, and the wireless transmission box sends the received image signal to the first display screen through the antenna. The remote control receiving module includes a remote control receiver and an antenna. The remote control receiver and the remote control handle exchange data through radio communication. The remote control receiver sends the state parameters of the machine received from the controller, such as battery voltage, coolant temperature, hydraulic oil temperature, engine speed, pilot state, gear state and other data to the remote control handle through the antenna, which can be displayed on the LCD display screen on the remote control handle; the remote control handle collects the operation signals of the joystick, knob, switch and the like on the handle, and transmits them to the antenna of the remote control receiver, and then transmits them to the remote control receiver.

[0050] The controller and the remote control receiver exchange data through the CAN bus. The controller sends the machine's status parameters, such as battery voltage, coolant temperature, hydraulic oil temperature, engine speed, pilot status, gear status and other data to the remote control receiver; the remote control receiver sends the received remote control data, such as foot pedal, operating lever, ignition, flameout, pilot switch, horn, lighting, throttle gear, driving mode, etc., to the controller, and then the controller controls the construction machinery to execute corresponding instructions or actions.

[0051] In some embodiments, the engineering machine is further provided with a second display, and a signal input terminal of the second display is connected to a signal output terminal of the controller.

[0052] In this embodiment, the above-mentioned second display can be set in the cab, and the controller displays the status parameters of the machine on the second display screen to facilitate viewing the status parameters on the construction machinery, realize remote control operation of the construction machinery at a distance of not less than 150 meters, and at the same time obtain the status of the construction machinery.

[0053] In some embodiments, a signal output terminal of the image processing host is connected to a signal input terminal of the second display.

[0054] In this embodiment, the image processing host can output and display the video images collected by cameras in multiple directions and the generated 540-degree panoramic image (transparent chassis) and lidar point cloud map to the second display screen to facilitate viewing of video information and the three-dimensional size and position distance information of the object.

[0055] Taking an excavator as an example of construction machinery, the excavator itself is equipped with an excavator controller and a cab display. The excavator controller is connected to the remote control receiver through the CAN bus for data exchange; the cab display can receive and display the 540-degree panoramic image generated by the image processing host, the video images collected by each camera, and the lidar point cloud map.

[0056] Taking the construction machinery as an excavator as an example, the data flow of the solution is explained. Please refer to Figure 1 ,

[0057] Arrow 1: represents the excavator controller displaying the machine's status parameters on the cab display screen.

[0058] Arrow 2: represents that the image processing host outputs and displays the video images collected by cameras in four directions and the generated 540-degree panoramic image (transparent chassis) and lidar point cloud map to the cab display screen.

[0059] Arrow 3: represents that multiple high-definition cameras input the collected video image signals to the image processing host through the AHD data cable, and the laser radar inputs the collected point cloud data to the image processing host through the network cable.

[0060] Arrow 4: represents the data exchange between the excavator controller and the remote control receiver via the CAN bus. The excavator controller sends the state parameters of the machine, such as battery voltage, coolant temperature, hydraulic oil temperature, engine speed, pilot state, gear state, etc., to the remote control receiver; the remote control receiver sends the received remote control data, such as pedal, operating lever, ignition, flameout, pilot switch, horn, lighting, throttle gear, driving mode, etc., to the excavator controller, and then the controller controls the excavator to execute the corresponding instructions or actions.

[0061] Arrow 5: represents that the image processing host transmits the video image signals collected by the cameras in four directions and the generated panoramic image and lidar point cloud map to the wireless transmitter box through the AHD data cable.

[0062] Arrow 6 and arrow 8: represent the data exchange between the remote control receiver and the remote control handle through radio communication. The remote control receiver sends the state parameters of the machine received from the excavator controller, such as battery voltage, coolant temperature, hydraulic oil temperature, engine speed, pilot state, gear state and other data to the remote control handle through the antenna, and displays it on the LCD screen on the remote control handle; the remote control handle collects the operation signals of the joystick, knob, switch and other operation signals on the handle, transmits them to the antenna of the remote control receiver, and then transmits them to the remote control receiver.

[0063] Arrow 7 and Arrow 9: represent that the wireless transmitter box sends the received image signal to the wireless display screen through the antenna.

[0064] Arrow 10: represents an independent mobile power supply to the remote control handle.

[0065] Arrow 11: represents an independent mobile power supply to the wireless display screen.

[0066] Arrow 12: Indicates that the operator can operate the remote control handle and can also observe the LCD display on the handle to obtain the status parameters of the excavator.

[0067] Arrow 13: Indicates that the operator can observe the video images and 540-degree panoramic images taken by the cameras in four directions, as well as the LiDAR point cloud map on the wireless display screen. The operator can also adjust the brightness of the image and the display mode, for example, setting the display to two screens, respectively displaying the front camera monitoring video and the LiDAR point cloud map.

[0068] Based on the above scheme, the excavator short-range remote control system can achieve the following functions:

[0069] a) Realize remote control excavator operation at a distance of not less than 150 meters, and obtain the status of the excavator at the same time.

[0070] b) Realize 540-degree wireless panoramic video image monitoring at a distance of no less than 150 meters, and also have the functions of human body recognition and distance detection.

[0071] c) Realize wireless LiDAR point cloud monitoring function with a distance of no less than 150 meters, and can sense the three-dimensional size and position distance of objects.

[0072] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0073] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A remote control system for construction machinery, characterized in that: include: A first display screen and a remote control handle provided at the remote control end; An image processing host, a wireless communication module, a laser radar device and a plurality of camera devices installed on the engineering machinery; The remote control handle is used to remotely control the engineering machinery. The signal output ends of the laser radar equipment and multiple camera equipment are respectively connected to the signal input ends of the image processing host. The signal output end of the image processing host is connected to the signal input end of the first display screen through the wireless communication module.

2. The construction machinery remote control system according to claim 1, characterized in that: The remote control end is also provided with a fixing bracket, and the display screen is installed on the remote control handle through the fixing bracket.

3. The engineering machinery remote control system according to claim 2, characterized in that: The angle of the fixing bracket is adjustable.

4. The engineering machinery remote control system according to claim 1, characterized in that: The remote control handle is provided with a shoulder strap.

5. The engineering machinery remote control system according to claim 1, characterized in that: The engineering machine is provided with a controller, and the controller transmits data with the remote control handle through the wireless communication module.

6. The engineering machinery remote control system according to claim 5, characterized in that: The wireless communication module includes a wireless transmission module and a remote control receiving module. The signal output end of the image processing host is connected to the signal input end of the wireless transmission module, and the signal output end of the wireless transmission module is connected to the signal input end of the first display screen; the controller is connected to the remote control receiving module, and the remote control receiving module is connected to the remote control handle.

7. The engineering machinery remote control system according to claim 5, characterized in that: The engineering machine is also provided with a second display, and a signal input end of the second display is connected to a signal output end of the controller.

8. The engineering machinery remote control system according to claim 7, characterized in that: The signal output terminal of the image processing host is connected to the signal input terminal of the second display.

9. The engineering machinery remote control system according to claim 1, characterized in that: A sun hat is arranged on the first display screen.

10. The engineering machinery remote control system according to claim 1, characterized in that: The signal output end of each camera device is connected to the signal input end of the image processing host through an AHD data line.

Images