Remote driving system of underground coal mine vehicle

Abstract

The invention discloses a remote driving system for an underground coal mine vehicle, and the system comprises a ground remote control terminal which comprises a remote driving cabin, a server, a dynamic platform, a modularized armrest box, a first loudspeaker, and a first microphone; the server, the dynamic platform, the modularized armrest box, the loudspeaker I and the microphone I are all connected with the remote cockpit; the underground remote controlled vehicle is connected with the remote cockpit through underground communication equipment, and the underground remote controlled vehicle is provided with an intrinsic safety vehicle-mounted camera, an intrinsic safety IMU, a UWB positioning card, a vehicle sensor, a remote vehicle end controller, a second loudspeaker and a second microphone. The underground vehicle can be remotely controlled on the ground, the underground complex environment can be dealt with, the actual operation feeling of a driver is improved, and the driving safety of the underground remotely-controlled vehicle is guaranteed. The remote driving system has the advantages of high driving feeling, high safety and high suitability.

Description

Technical Field

[0001] The present invention relates to the technical field of underground vehicles, and in particular to a remote driving system for underground vehicles in coal mines. Background Art

[0002] Due to the harsh environment in coal mines, the mining process is still relatively backward and the degree of intelligence is not high. At present, many jobs still rely on manual work. For example, underground transport vehicles are difficult for drivers to drive and the workload is relatively high due to the complex underground tunnels, harsh environment, and dim light. In addition, there are harmful gases, dust, noise, etc. in the underground environment, which pose a great threat to the physical and mental health of drivers.

[0003] Therefore, the present invention proposes a remote driving system suitable for an underground vehicle, and a driver can perform remote driving operations on the underground vehicle on the ground. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a remote driving system for underground vehicles in a coal mine.

[0005] The technical solution adopted by the present invention to solve its technical problems is: a remote driving system for underground vehicles in a coal mine, comprising: a ground remote control terminal, comprising: a remote cockpit, a server, a dynamic platform, a modular armrest box, a speaker one and a microphone one; the server, the dynamic platform, the modular armrest box, the speaker one and the microphone one are all connected to the remote cockpit; an underground remotely controlled vehicle, the underground remotely controlled vehicle is connected to the remote cockpit through an underground communication device, and the underground remotely controlled vehicle is equipped with an intrinsically safe vehicle-mounted camera, an intrinsically safe IMU, a UWB positioning card, the vehicle's own sensor, a remote vehicle-end controller, a speaker two and a microphone two.

[0006] Furthermore, camera No. 1, camera No. 2, camera No. 3, camera No. 4, camera No. 5 and camera No. 6 are installed on the left front, front, right front, right rear, front and left rear of the underground remote controlled vehicle respectively, and the server receives the video images taken by camera No. 1, camera No. 2, camera No. 3, camera No. 4, camera No. 5 and camera No. 6 in real time and synchronously.

[0007] Furthermore, a No. 1 millimeter-wave radar, a No. 2 millimeter-wave radar, a No. 3 millimeter-wave radar, a No. 4 millimeter-wave radar, a No. 5 millimeter-wave radar, and a No. 6 millimeter-wave radar are respectively installed at the front, rear, front left, rear left, front right, and rear right of the underground remote-controlled vehicle. The No. 1 millimeter-wave radar is integrated with the No. 1 camera through a tooling bracket, the No. 2 millimeter-wave radar is integrated with the No. 2 camera through a tooling bracket, the No. 3 millimeter-wave radar is integrated with the No. 3 camera through a tooling bracket, the No. 4 millimeter-wave radar is integrated with the No. 4 camera through a tooling bracket, the No. 5 millimeter-wave radar is integrated with the No. 5 camera through a tooling bracket, and the No. 6 millimeter-wave radar is integrated with the No. 6 camera through a tooling bracket.

[0008] Furthermore, six millimeter-wave radars are used to detect radar data of different directions of the vehicle and obstacles. The radar data is processed by the remote vehicle-side controller into obstacle distance data at the corresponding position and then transmitted to the remote cockpit. The video image is spliced ​​by the server to form a 360° panoramic view of the vehicle body and transmitted to the remote cockpit. The video image corresponds one-to-one to the obstacle distance data.

[0009] Furthermore, assume that the obstacle distance information of the front, rear, left front, left rear, right front and right rear of the underground remote-controlled vehicle are df, dr, da, dc, db and dd respectively. When the vehicle is driving, if the obstacle distance information is less than the first-level warning distance, the corresponding position of the display screen of the remote cockpit will show a yellow backlight with a low-frequency flashing prompt; if the obstacle distance information is less than the second-level warning distance, the corresponding position of the display screen of the remote cockpit will show a red backlight with a low-frequency flashing prompt, and a voice broadcast warning will be given through the speaker one.

[0010] Furthermore, the video image splicing process includes:

[0011] S1, after receiving the video images captured by cameras No. 1 to No. 6, the server extracts pixel features of the edges of all video images;

[0012] S2, extracting edge pixel feature points of the current video image and edge pixel feature points of video images on both sides adjacent to the video image, and identifying the same edge feature points of the video image and the video images adjacent to it;

[0013] S3, performing overlapping alignment on the same edge feature point regions to splice the current video image screen with the adjacent video image screen;

[0014] S4. All video images are stitched together to obtain a 360° panoramic view of the vehicle body.

[0015] Furthermore, the display screen of the remote cockpit displays the underground communication quality in real time, and adjusts the maximum speed of the underground remote-controlled vehicle according to the underground communication quality; when the time delay of the underground communication quality is less than 100ms, the maximum speed of the underground remote-controlled vehicle is limited to ≤20km / h; when the time delay of the underground communication quality is between 100ms and 200ms, the maximum speed of the underground remote-controlled vehicle is limited to ≤10km / h; when the time delay of the underground communication quality is between 200ms and 400ms, the maximum speed of the underground remote-controlled vehicle is limited to ≤5km / h; when the time delay of the underground communication quality is greater than 400ms, remote driving is stopped.

[0016] Furthermore, a No. 7 camera is installed inside the underground remote controlled vehicle, and the shooting angle of the No. 7 camera faces the front of the vehicle.

[0017] Furthermore, the server records the IP number of each underground remote-controlled vehicle and the corresponding operation configuration; when the driver enters the remote cockpit, he selects the IP number of the vehicle to be operated to operate the corresponding underground remote-controlled vehicle.

[0018] Furthermore, the intrinsically safe IMU can collect the acceleration, pitch angle, and heading angle of the underground remote-controlled vehicle in real time during driving. The dynamic platform makes corresponding gestures according to the data collected by the intrinsically safe IMU so that the driver can feel the driving condition of the underground remote-controlled vehicle more intuitively.

[0019] The beneficial effects of the present invention are:

[0020] It has a 360-degree bird's-eye view and safety warning line function to ensure safe driving distance perception;

[0021] The intrinsically safe vehicle-mounted camera with cleaning and infrared fill-in light functions can perform infrared fill-in light when the underground tunnel is insufficiently illuminated, and can perform lens cleaning when the lens is blurred;

[0022] Added a front-facing in-car perspective (the installation position is inside the car, and the perspective inside and in front of the car can be observed at the same time), which is equivalent to adding a third-person perspective option. The main perspective can switch between the front and in-car perspectives, which can be used as a reference for ground drivers, effectively alleviating dizziness and improving judgment reliability;

[0023] The intrinsically safe IMU collects information such as acceleration, pitch angle, and heading angle during the operation of the vehicle. The dynamic platform of the ground cockpit uses this information as a basis to make corresponding gestures, so that the ground remote control driver can more intuitively feel the actual driving conditions of the underground vehicle and make more accurate and effective judgments.

[0024] The server can record the IP number of each vehicle that needs remote control and the relevant wipe operation configuration of each vehicle;

[0025] Ability to limit vehicle speed based on network communication quality;

[0026] It is equipped with a remote network intercom function (speaker and microphone). In case of emergency, this function can be used to enable voice intercom between the remote driver and the passengers and staff around the vehicle, ensuring a communication bridge between the vehicle and the ground cabin. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0028] Figure 1 It is an architectural diagram of the remote driving system of the present invention.

[0029] Figure 2 It is a schematic diagram of the installation of cameras No. 1 to No. 6 / millimeter wave radars of the present invention (top view).

[0030] Figure 3 It is a schematic diagram of the camera and millimeter wave radar of the present invention installed as one.

[0031] Figure 4 It is a schematic diagram of the distance between the vehicle and obstacles at different positions of the present invention. DETAILED DESCRIPTION

[0032] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0033] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0034] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0035] like Figures 1 to 3 As shown, the remote driving system of underground vehicles in a coal mine of the present invention comprises: a ground remote control terminal and an underground remote controlled vehicle, wherein the ground remote control terminal comprises: a remote cockpit, a server, a dynamic platform, a modular armrest box, a speaker 1 and a microphone 1; the server, the dynamic platform, the modular armrest box, the speaker 1 and the microphone 1 are all connected to the remote cockpit. The underground remote controlled vehicle is connected to the remote cockpit through underground communication equipment, and the underground remote controlled vehicle is equipped with an intrinsically safe vehicle-mounted camera, an intrinsically safe IMU, a UWB positioning card, a vehicle's own sensor, a remote vehicle controller, a speaker 2 and a microphone 2.

[0036] It should be noted that the remote cockpit is mainly used for remote control of underground remote controlled vehicles. The server can record the IP number of each underground remote controlled vehicle and the operation configuration of the object. For example, when the underground remote controlled vehicle is a loader, in addition to the normal remote driving configuration, bucket control needs to be added; when a certain vehicle underground needs to be remotely driven, the driver selects the IP number of the vehicle in the remote cockpit, and the cockpit automatically switches to the corresponding operation configuration of the vehicle; at the same time, the server can record the relevant data of remote driving, including: remote time, fault information, history, issued instructions, etc., support data export, and facilitate management. In addition to the normal steering wheel, accelerator, brake, and display screen, the remote cockpit adds a modular armrest box with joysticks, switches, etc. Through the UWB positioning card on the vehicle, the underground UWB system can be used to build a one-dimensional map of the underground, display the location of the underground remote controlled vehicle in real time, and display it on the display screen of the remote cockpit, so that the driver can judge the location of the vehicle underground. The intrinsically safe IMU (inertial sensor) can collect information such as acceleration, pitch angle, heading angle, etc. during the driving process of the remote controlled vehicle underground in real time. The dynamic platform makes corresponding gestures based on the data collected by the intrinsically safe IMU, so that the ground driver can more intuitively feel the actual driving conditions of the underground vehicle and make more accurate and effective judgments. The intrinsically safe vehicle-mounted camera has the functions of cleaning and infrared fill light. When the lighting in the underground tunnel is insufficient, infrared fill light is used; when the dust and water mist are large and the lens is blurred, the lens is automatically cleaned. Through the speaker and microphone, it has a remote network intercom function. When an emergency occurs, this function can be used to realize voice intercom between the remote driver and the passengers on the underground vehicle and the staff around the vehicle, ensuring the communication bridge between the vehicle end and the ground cabin end.

[0037] Due to the complex terrain, dim light, narrow lanes and other factors in the underground environment, remote driving is difficult to a certain extent. Therefore, the present invention installs camera No. 1, camera No. 2, camera No. 3, camera No. 4, camera No. 5, and camera No. 6 on the left front, front, right front, right rear, front, and left rear of the remote controlled vehicle underground, and the server receives the video images taken by camera No. 1, camera No. 2, camera No. 3, camera No. 4, camera No. 5, and camera No. 6 in real time and synchronously. The six cameras are used to shoot pictures in different directions around the vehicle, and the video images shot by each camera can be transmitted to the server on the ground through the remote vehicle controller. The display screen of the remote cockpit can display the front view picture, left view picture (switchable front and back), right view picture (switchable front and back), rear view picture, inner view picture, 360° surround view picture, UWB positioning map, communication quality data, vehicle data, etc. Millimeter-wave radar No. 1, millimeter-wave radar No. 2, millimeter-wave radar No. 3, millimeter-wave radar No. 4, millimeter-wave radar No. 5 and millimeter-wave radar No. 6 are installed in front, rear, front left, rear left, front right and rear right of the remote-controlled vehicle underground respectively. The No. 1 millimeter-wave radar is integrated with camera No. 1 through a tooling bracket, the No. 2 millimeter-wave radar is integrated with camera No. 2 through a tooling bracket, the No. 3 millimeter-wave radar is integrated with camera No. 3 through a tooling bracket, the No. 4 millimeter-wave radar is integrated with camera No. 4 through a tooling bracket, the No. 5 millimeter-wave radar is integrated with camera No. 5 through a tooling bracket, and the No. 6 millimeter-wave radar is integrated with camera No. 6 through a tooling bracket.

[0038] Six millimeter-wave radars are used to detect radar data of different positions of the vehicle and obstacles. The radar data is processed by the remote vehicle controller into obstacle distance data at the corresponding position and then transmitted to the remote cockpit. The video image is spliced ​​and processed by the server to form a 360-degree surround view of the vehicle body and transmitted to the remote cockpit. The video image corresponds to the obstacle distance data one by one. In other words, when the 360-degree surround view is displayed on the display screen of the remote cockpit, the distance information between each position of the vehicle and the obstacle will be displayed at the same time, which is convenient for the driver to adjust the driving strategy in time.

[0039] like Figure 4As shown in the figure, let the obstacle distance information in the due front, due rear, front left, rear left, front right, and rear right of the underground remotely controlled vehicle be df, dr, da, dc, db, and dd respectively. The obstacles on the left and right sides of the underground remotely controlled vehicle are mainly roadway walls, and the obstacles in the front and rear of the underground remotely controlled vehicle may be staff, equipment, etc. Through the video images and 360° panoramic views in each direction and the obstacle distance data, the driver in the remote cockpit can intuitively understand whether there are obstacles around the current position of the underground remotely controlled vehicle and the distance to the obstacles. For example, when the vehicle is moving forward, the image displayed in the left view of the remote cockpit is the video data transmitted back in real time by the first camera, and the distance da is also displayed; the image displayed in the due front view of the remote cockpit is the video data transmitted back in real time by the second camera, and the distance df is also displayed; the image displayed in the front right view of the remote cockpit is the video data transmitted back in real time by the third camera, and the distance db is also displayed. When the vehicle is moving backward, the left front right views of the remote cockpit display the video data transmitted back in real time by the sixth, fifth, and fourth cameras respectively, and the distances dc, dr, and dd are also displayed.

[0040] In addition, considering the safety of the underground remotely controlled vehicle during driving, and at the same time to enhance the intuitive feeling of the driver in the remote cockpit about the distance from the underground roadway wall edge and obstacles, two-level alarms are set for the distance between the underground remotely controlled vehicle and the underground roadway wall edge and obstacles during driving. If the obstacle distance information < the first-level warning distance, the corresponding position of the display screen in the remote cockpit will be prompted by a low-frequency flash of yellow backlight; if the obstacle distance information < the second-level warning distance, the corresponding position of the display screen in the remote cockpit will be prompted by a low-frequency flash of red backlight, and a voice alarm will be broadcast through the first horn.

[0041] For example, taking the warning on the left side of the vehicle as an example, the first-level alarm: if da < DA1, the image displayed in the left view of the remote cockpit will be prompted by a low-frequency flash of yellow backlight to remind the driver in the remote cockpit that the distance from the left front of the underground vehicle to the roadway wall edge is relatively close; the second-level alarm: da < DA2, the image displayed in the left view of the remote cockpit will be prompted by a high-frequency flash of red backlight, and at the same time, the first horn of the remote control cabin on the ground will broadcast the warning information "Front left alarm!", to remind the driver to take measures immediately to avoid vehicle collision. At the same time, the underground vehicle itself will take braking measures to control the vehicle speed ≤ 5 km / h. The specific values of DA1 and DA2 can be set according to the actual width of the roadway and the width of the vehicle body. Calculated with a vehicle width of 2 m and a roadway width of 5 m, generally DA1 is set to 1 m and DA2 is set to 0.5 m, that is, DA1 is 0.5 vehicle widths, DA2 is 0.5 DA1, and the values of DA1 and DA2 can be flexibly set according to the on-site implementation situation. The setting principle is that DA2 must ensure that the driver in the remote cockpit can avoid vehicle collision accidents under the current vehicle speed and network support conditions.

[0042] Since da, db, dd, and dc are all obstacle distance measurements at the lateral level of the vehicle, the two-level alarm distances DA1, DA2, DB1, DB2, DD1, DD2, DC1, and DC2 are set in the same principle as in the above example (the lateral moving speed of vehicles in coal mines is generally <5km / h). Since the maximum longitudinal moving speed of vehicles in coal mines is generally <5km / h when they are reversing, the two-level alarm distances DR1 and DR2 at the dr level are set in the same principle as in the above example. Here, the two-level alarm distances DF1 and DF2 of df need to be considered separately. Since the quality of underground communication may change and signal delay may occur, the maximum speed of the remotely controlled vehicle in the underground needs to be limited. For example, under a network delay of 100ms, the maximum speed of a remotely driven underground vehicle is no more than 20km / h, corresponding to a vehicle speed of approximately 5.56m / s, and a corresponding uncontrolled driving distance of 0.556m. Taking a certain type of rubber-tyred trackless mining vehicle as an example, two-level alarm distances DF1 and DF2 are set. At an initial speed of 20km / h, the maximum braking distance of cold braking is ≤6m, and the maximum braking distance of hot braking is ≤7.5m. At this time, considering the vehicle braking performance and the impact of network infrastructure, after the No. 2 millimeter-wave radar detects the distance to the obstacle in front of the vehicle, the safety distance required for the driver in the remote cockpit to take safety measures needs to be greater than 8.056m. Considering the reaction time required by the driver, DF2 is set to 10m, and DF1 is set to 2DF2, that is, DF1 is set to 20m.

[0043] For example, the stitching process of the video images of six cameras includes: S1, after the server receives the video images captured by cameras No. 1 to No. 6, it extracts the pixel point features of the edges of all video images; S2, extracts the edge pixel feature points of the current video image and the edge pixel feature points of the video images on both sides adjacent to the video image, and identifies the same edge feature points of the video image and the video images adjacent to it; S3, aligns the areas with the same edge feature points to overlap and stitch the current video image screen with the adjacent video image screen; S4, after stitching all the video images, a 360° surround view of the vehicle body is obtained.

[0044] Specifically, after receiving the video image of camera No. 1, the server extracts the RGB pixel features of the left edge of the video image of camera No. 1 in real time, and extracts the RGB pixel features of the right edge of the video image of camera No. 6 at the same time, and then matches the RGB pixel features of the left edge of the video image of camera No. 1 with the right edge of the video image of camera No. 6, and overlaps and aligns the area with the same RGB pixel features of the left edge of the video image of camera No. 1 and the right edge of the video image of camera No. 6, thus completing the splicing work between the video image of camera No. 1 and the video image of camera No. 6. At the same time, the RGB pixel features of the right edge of the video image of camera No. 1 are extracted in real time, and the RGB pixel features of the left edge of the video image of camera No. 2 are extracted at the same time, and then the RGB pixel features of the right edge of the video image of camera No. 1 and the left edge of the video image of camera No. 2 are matched, and the area with the same RGB pixel features of the right edge of the video image of camera No. 1 and the left edge of the video image of camera No. 2 are overlapped and aligned, thus completing the splicing work between the video image of camera No. 1 and the video image of camera No. 2. The splicing process of the remaining video images of cameras No. 2 to No. 6 is similar, and a 360° panoramic view is finally formed after all the splicing is completed.

[0045] For example, safety warning lines (corresponding to safety distances of 3, 2, and 1 meters) are also marked on the display screen of the remote cockpit. After the controlled vehicle has calibrated the lane lines with the front and rear cameras, the static lane line reference for driving can be turned on. The lane lines are red, yellow, and green from near to far, and the distance between the lane lines is 1 meter.

[0046] Since the remote control signal is transmitted through the underground communication equipment, the communication quality of the underground communication equipment (including uplink and downlink bandwidth, communication delay, etc.) may not be able to maintain a good state all the time. When the communication quality is not very good, the control effect of the underground remote controlled vehicle will also change, which may cause an accident in serious cases. Based on this, the present invention can adjust the maximum speed of the underground remote controlled vehicle according to the underground communication quality. Specifically, when the delay of the underground communication quality is less than 100ms, the maximum speed of the underground remote controlled vehicle is limited to ≤20km / h; when the delay of the underground communication quality is between 100ms and 200ms, the maximum speed of the underground remote controlled vehicle is limited to ≤10km / h; when the delay of the underground communication quality is between 200ms and 400ms, the maximum speed of the underground remote controlled vehicle is limited to ≤5km / h; when the delay of the underground communication quality is greater than 400ms, stop remote driving.

[0047] It should be noted that the communication delay can be obtained in real time through the ping tool software. 100ms is 0.1s. At this time, the maximum speed of the remote-driven underground vehicle is no more than 20km / h, and the corresponding vehicle speed is about 5.56m / s, that is, the maximum uncontrollable moving distance of the vehicle corresponding to 100ms is 0.556m. Combined with the aforementioned warning mechanism function, the vehicle is in a controllable state at this time. 200ms is 0.2s. At this time, the maximum speed of the remote-driven underground vehicle is no more than 10km / h, and the corresponding vehicle speed is about 2.78m / s, that is, the maximum uncontrollable moving distance of the vehicle corresponding to 200ms is 0.556m. Combined with the aforementioned warning mechanism function, the vehicle is in a controllable state at this time. 400ms is 0.4s. At this time, the maximum speed of the remote-driven underground vehicle is no more than 5km / h, and the corresponding vehicle speed is about 1.39m / s, that is, the maximum uncontrollable moving distance of the vehicle corresponding to 400ms is 0.556m. Combined with the aforementioned warning mechanism function, the vehicle is in a controllable state at this time. When the delay is greater than 400ms, the remote driving experience is poor and remote driving is prohibited to avoid accidents.

[0048] The maximum speed limit of the remote-driven vehicle corresponding to the network delay in the present invention is only an embodiment. In actual application, the maximum speed limit of the remote-driven vehicle corresponding to the network delay can be flexibly set according to the on-site implementation situation. The setting principle is that DA2 must ensure that the current vehicle speed and the remote cockpit driver can avoid vehicle collision accidents under network support conditions. If the driving speed of the underground remote controlled vehicle exceeds the maximum speed under the condition of network delay, a collision accident is likely to occur. For example, under a delay of 100ms, the maximum speed of the underground remote controlled vehicle is greater than 20km / h, and the corresponding vehicle speed is about 5.56m / s, that is, the maximum uncontrollable moving distance of the vehicle corresponding to the delay of 100ms exceeds 0.556m. If the communication delay causes the driving route of the underground remote-driven vehicle to deviate, it is easy to collide with the wall of the tunnel. At this time, the vehicle will trigger the first level alarm, and the remote cockpit driver will receive the flashing state of the relevant screen and correct the driving direction in time to avoid collision. If the driver in the remote cockpit receives the flashing status of the relevant screen and fails to correct the driving direction in time, when the distance from the obstacle in each direction reaches the second-level alarm distance, the vehicle will execute the second-level alarm measures.

[0049] A No. 7 camera is installed inside the underground remote-controlled vehicle, and the shooting angle of the No. 7 camera faces the front of the vehicle. The No. 7 camera is fixed inside the underground remote-controlled vehicle through tooling. The No. 7 camera is used to collect video images from the inside of the vehicle and the front of the vehicle. Using the video image of the No. 2 camera as the main visual basis for remote control is similar to the first-person perspective in the game. After a long time, some drivers are prone to dizziness, resulting in inaccurate judgments and other phenomena. Adding an in-vehicle perspective facing the front (the installation position is located inside the vehicle, and the perspectives inside the vehicle and in front of the vehicle can be observed at the same time) is equivalent to adding a third-person perspective option. The remote driver can refer to the video image of the No. 7 camera, which effectively alleviates the dizziness problem and improves the reliability of judgment.

[0050] The ground cockpit obtains real-time images, UWB positioning map data, communication delay, and body-related data collected by sensors and controllers installed on the underground vehicle body through underground 5G and other communication networks, and displays them on the display screen of the remote cockpit. The cockpit issues control commands to control the movement of the underground vehicle in real time through the communication network. This system also has the function of remote ignition and shutdown, which can start and stop the vehicle remotely without manual start and stop.

[0051] In summary, the remote driving system of underground coal mine vehicles of the present invention has multiple functions, can remotely control underground vehicles on the ground, can cope with the complex environment underground, improve the actual operation experience of the driver, and ensure the driving safety of the remotely controlled vehicles underground. The remote driving system of the present invention has the advantages of high driving feeling, high safety and high adaptability.

[0052] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A remote driving system for underground vehicles in coal mines, characterized in that: include: The ground remote control terminal includes: a remote cockpit, a server, a dynamic platform, a modular armrest box, a speaker and a microphone; The server, the dynamic platform, the modular armrest box, the speaker 1 and the microphone 1 are all connected to the remote cockpit; An underground remote-controlled vehicle, the underground remote-controlled vehicle is connected to the remote cockpit through underground communication equipment, and the underground remote-controlled vehicle is equipped with an intrinsically safe vehicle-mounted camera, an intrinsically safe IMU, a UWB positioning card, the vehicle's own sensors, a remote vehicle-side controller, two speakers and two microphones.

2. The remote driving system for underground vehicles in coal mines according to claim 1, characterized in that: Camera No. 1, camera No. 2, camera No. 3, camera No. 4, camera No. 5 and camera No. 6 are installed on the left front, front, right front, right rear, front and left rear of the underground remote controlled vehicle respectively, and the server receives the video images taken by camera No. 1, camera No. 2, camera No. 3, camera No. 4, camera No. 5 and camera No. 6 in real time and synchronously.

3. The remote driving system for underground vehicles in coal mines according to claim 2, characterized in that: A No. 1 millimeter-wave radar, a No. 2 millimeter-wave radar, a No. 3 millimeter-wave radar, a No. 4 millimeter-wave radar, a No. 5 millimeter-wave radar and a No. 6 millimeter-wave radar are respectively installed in the front, rear, front left, rear left, front right and rear right of the underground remote-controlled vehicle. The No. 1 millimeter-wave radar is integrated with the No. 1 camera through a tooling bracket, the No. 2 millimeter-wave radar is integrated with the No. 2 camera through a tooling bracket, the No. 3 millimeter-wave radar is integrated with the No. 3 camera through a tooling bracket, the No. 4 millimeter-wave radar is integrated with the No. 4 camera through a tooling bracket, the No. 5 millimeter-wave radar is integrated with the No. 5 camera through a tooling bracket, and the No. 6 millimeter-wave radar is integrated with the No. 6 camera through a tooling bracket.

4. The remote driving system for underground vehicles in coal mines according to claim 3, characterized in that: Six millimeter-wave radars are used to detect radar data of different directions of the vehicle and obstacles. The radar data is processed by the remote vehicle-side controller into obstacle distance data at the corresponding position and then transmitted to the remote cockpit. The video image is spliced ​​by the server to form a 360-degree panoramic view of the vehicle body and transmitted to the remote cockpit. The video image corresponds one-to-one to the obstacle distance data.

5. The remote driving system for underground vehicles in coal mines according to claim 4, characterized in that The obstacle distance information of the front, rear, left front, left rear, right front and right rear of the underground remote controlled vehicle are df, dr, da, dc, db and dd respectively. When the vehicle is driving, if the obstacle distance information is less than the first-level warning distance, the corresponding position of the display screen of the remote cockpit will show a yellow backlight with a low-frequency flashing prompt; If the obstacle distance information is less than the secondary warning distance, the corresponding position of the display screen of the remote cockpit will be flashed with a red backlight at a low frequency, and a voice broadcast warning will be given through the speaker 1.

6. The remote driving system for underground vehicles in coal mines according to claim 4, characterized in that: The video image splicing process includes: S1, after receiving the video images captured by cameras No. 1 to No. 6, the server extracts pixel features of the edges of all video images; S2, extracting edge pixel feature points of the current video image and edge pixel feature points of video images on both sides adjacent to the video image, and identifying the same edge feature points of the video image and the video images adjacent to it; S3, performing overlapping alignment on the same edge feature point regions to splice the current video image screen with the adjacent video image screen; S4. All video images are stitched together to obtain a 360° panoramic view of the vehicle body.

7. The remote driving system for underground vehicles in a coal mine as claimed in claim 1, characterized in that: The display screen of the remote cockpit displays the underground communication quality in real time, and adjusts the maximum speed of the underground remote controlled vehicle according to the underground communication quality; When the time delay of underground communication quality is less than 100ms, the maximum speed of the underground remote-controlled vehicle is limited to ≤20km / h; when the time delay of underground communication quality is between 100ms and 200ms, the maximum speed of the underground remote-controlled vehicle is limited to ≤10km / h; when the time delay of underground communication quality is between 200ms and 400ms, the maximum speed of the underground remote-controlled vehicle is limited to ≤5km / h; when the time delay of underground communication quality is greater than 400ms, remote driving is stopped.

8. The remote driving system for underground vehicles in coal mines according to claim 1, characterized in that: A No. 7 camera is installed inside the underground remote controlled vehicle, and the shooting angle of the No. 7 camera faces the front of the vehicle.

9. The remote driving system for underground vehicles in a coal mine as claimed in claim 1, characterized in that: The server records the IP number of each underground remote-controlled vehicle and the corresponding operation configuration; when the driver enters the remote cockpit, he selects the IP number of the vehicle to be operated and can operate the corresponding underground remote-controlled vehicle.

10. The remote driving system for underground vehicles in a coal mine according to claim 1, characterized in that: The intrinsically safe IMU can collect the acceleration, pitch angle and heading angle of the underground remote-controlled vehicle in real time during driving. The dynamic platform makes corresponding gestures according to the data collected by the intrinsically safe IMU so that the driver can feel the driving condition of the underground remote-controlled vehicle more intuitively.

Images