Unmanned mine car remote control driving method and device

Abstract

The invention provides an unmanned mine car remote control driving method and device, and the method comprises the steps: responding to a remote control takeover signal, and drawing a three-dimensional simulation map; mapping the model of the unmanned mine car to a three-dimensional simulation map; driving assistance information is generated according to the surrounding environment information and drawn on a three-dimensional simulation map; the driving assistance information comprises obstacle information and / or V2V vehicle information; based on the driving assistance information, generating a guide path according to the current position information and the target position information of the unmanned mine car, and drawing the guide path on a three-dimensional simulation map; and controlling the unmanned mine car to move along the guide path according to the control instruction. The remote control takeover mode not only improves the driving experience, but also enhances the safety and reliability of the unmanned mine car.

Description

Technical Field

[0001] The present application relates to the field of unmanned driving technology, and more specifically, to a method and device for remotely controlling an unmanned mining vehicle. Background Art

[0002] As unmanned mining vehicles are put into normal operation in mining areas, if unmanned mining vehicles stop due to malfunctions or obstacles, remote assistance is usually required to help them get out of trouble. At present, a remote operation center is usually built, and the vehicle collects sensor data, video and other information. These data are transmitted to the operation center via radio or satellite signals, and the staff remotely controls the unmanned mining vehicles to get out of trouble based on the information received.

[0003] However, in the related technology, driving is guided solely by images captured by the camera, which places high demands on the operator and requires analysis of the driving environment within a limited image field of view and a low image resolution, resulting in low efficiency and poor safety. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art.

[0005] To this end, the first aspect of the present application is to propose a remote control driving method for an unmanned mining vehicle.

[0006] The second aspect of the present application is to propose a remote control driving device for an unmanned mining vehicle.

[0007] In view of this, according to the first aspect of the present application, a method for remotely controlling an unmanned mining vehicle is proposed, comprising:

[0008] In response to the remote control takeover signal, draw a three-dimensional simulation map;

[0009] Map the model of the unmanned mining vehicle onto a three-dimensional simulation map;

[0010] Generate driving assistance information based on surrounding environment information and draw it on a three-dimensional simulation map; the driving assistance information includes obstacle information and / or V2V vehicle information;

[0011] Based on the driving assistance information, a guidance path is generated according to the current position information and target position information of the unmanned mining vehicle, and drawn on a three-dimensional simulation map;

[0012] The unmanned mining vehicle is controlled to move along the guide path according to the control instructions.

[0013] In the above technical solution, when the remote takeover system receives the remote control takeover signal, it is first necessary to draw a three-dimensional simulation map. Then the model of the unmanned mine car is mapped to this three-dimensional simulation map, so that the position of the unmanned mine car in the environment can be intuitively seen. The surrounding environment is perceived in real time by the sensors on the unmanned mine car, and driving assistance information is generated based on this information, and this information is drawn on the three-dimensional simulation map so that the operator can intuitively see this information; wherein the driving assistance information includes obstacle information and / or V2V (Vehicle-to-Vehicle, unmanned mine workshop communication technology) unmanned mine car information, so as to accurately reflect the environment around the unmanned mine car. After having the driving assistance information, a guidance path from the current position to the target position is generated according to the latitude and longitude of the current position of the unmanned mine car and the latitude and longitude of the target position, and it is drawn on the three-dimensional simulation map so that the operator can intuitively see how the unmanned mine car should travel to improve the efficiency and safety of the takeover. Finally, the unmanned mine car is controlled to drive safely along the guide path according to the control instructions generated by the operator. The control instructions are generated by the operator based on the guide path and driving assistance information, including steering, acceleration, deceleration, parking, etc. This remote control takeover method not only improves the driving experience, but also enhances the safety and reliability of the unmanned mine car.

[0014] In some technical solutions, optionally, in response to the remote control takeover signal, drawing a three-dimensional simulation map specifically includes:

[0015] In response to the remote control takeover signal, the latest map version number is obtained, and the information is verified with the local map version information;

[0016] If the latest map version number is inconsistent with the local map version information, download the latest map file and save it locally, and then verify the information again;

[0017] When the latest map version number and the local map version information are consistent, a three-dimensional simulation map is drawn based on the local map.

[0018] In the above embodiment, when the remote takeover system receives the remote control takeover signal, it will obtain the latest map version number from the server or cloud, and compare the latest map version number obtained with the map version information stored locally in the unmanned mine car to check whether there is a difference. If the latest map version number and the local map version information are inconsistent, it means that the local map needs to be updated. The remote takeover system will download the latest map file from a reliable source such as a server or cloud and save it locally for subsequent use; after the download is complete, the information is verified again to ensure that the downloaded map file is complete and correct. If the latest map version number and the local map version information are consistent, it means that the local map is already the latest version and does not need to be updated. Based on the locally stored map data, a three-dimensional simulation map is drawn to provide the operator with an intuitive driving environment view, which helps the operator better understand and control the driving environment of the unmanned mine car. By obtaining the latest map version number from the server or cloud, and performing version comparison and updating in real time, the real-time and accuracy of the map data used by the unmanned mine car are ensured.

[0019] In some technical solutions, optionally, drawing a three-dimensional simulated map based on a local map specifically includes: parsing a local map file and extracting two-dimensional vector map data; converting the two-dimensional vector map data into data in a three-dimensional space through coordinate projection conversion and vector data 3D projection, and drawing it on a real-time driving interface.

[0020] In the above technical solution, the local map file usually contains road geometry information, traffic signs and signals, topography and other data. By parsing the local map file, various two-dimensional vector data, such as road segments, polygons, points, etc., are extracted as the basis for constructing a three-dimensional simulation map. After completing the coordinate projection conversion, the two-dimensional vector map data is three-dimensionally projected through the vector data 3D projection, that is, it is converted into data in three-dimensional space, so that a realistic three-dimensional simulation map can be generated. Finally, the constructed three-dimensional simulation map is drawn on the real-time driving interface. The operator can intuitively see the driving environment around the unmanned mine car through this interface. Through precise coordinate projection conversion and three-dimensional modeling, the system can generate a realistic three-dimensional simulation map, providing the operator with an accurate view of the driving environment, which helps to improve the safety and accuracy of the remote control takeover of the unmanned mine car.

[0021] In some technical solutions, optionally, the obstacle information includes obstacle type and / or longitude and latitude; the V2V vehicle information includes at least one of the unmanned mining vehicle type, longitude and latitude, and heading angle.

[0022] In the above embodiment, the operator can make driving decisions such as acceleration, deceleration, steering, etc. according to obstacle information and V2V vehicle information to ensure driving safety and comfort and avoid collisions with other unmanned mining vehicles and obstacles.

[0023] In some technical solutions, optionally, when the driving assistance information includes obstacle information, the driving assistance information is generated according to the surrounding environment information and drawn on the three-dimensional simulation map, specifically including:

[0024] With the current position of the unmanned mining vehicle as the center, all obstacles within a certain radius are displayed; different icons are displayed for different obstacle types;

[0025] According to the current position of the unmanned mining vehicle and the preset distance, the obstacle display area is divided into multiple display areas; obstacles located in the same display area have the same display color, and obstacles located in different display areas have different display colors.

[0026] In the above technical solution, the current position of the unmanned mine car is first determined, and then a circular area is drawn with this position as the center. The radius of this circular area can be set as needed to cover obstacles within a certain range around the unmanned mine car. In order to distinguish different types of obstacles, different icons are used to represent them. This allows the operator to quickly identify the type of obstacle, improve the operator's perception of the surrounding environment, and reduce risks. Then, according to the current position of the unmanned mine car and the preset distance threshold, the circular obstacle display area is divided into multiple concentric rings or fan-shaped areas. Each area represents a different distance range, such as short distance, medium distance, and long distance, and different colors are used to represent obstacles located in different display areas. In this way, the operator can quickly judge the distance information of the obstacle by color.

[0027] In some technical solutions, optionally, the unmanned mine car remote control driving method also includes: generating safety fault information according to the unmanned mine car status information; and displaying the safety fault information to the user or operator.

[0028] In actual applications, once any abnormal fault or safety risk is detected, the corresponding safety fault information will be generated immediately, triggering the corresponding parking strategy, and reporting the safety fault information to the remote takeover system and displaying it to the user or operator, further improving the safety and reliability of remote control driving of unmanned mining vehicles.

[0029] In some technical solutions, optionally, the unmanned mine car remote control driving method further includes: when it is determined that the distance between the current position of the unmanned mine car and the obstacle meets a preset alarm threshold condition, an alarm reminder is issued.

[0030] The operator can quickly determine the distance to obstacles based on the alarm reminder, thereby making accurate driving decisions and improving driving efficiency.

[0031] In some technical solutions, optionally, the unmanned mine car remote control driving method also includes: acquiring a video stream of the surrounding environment of the unmanned mine car in real time; and calculating and splicing the video stream to generate a video picture.

[0032] In the above technical solution, based on the camera of the unmanned mine car, video images around the unmanned mine car are captured in real time, and these video streams are uploaded to the remote takeover system. In order to provide a more comprehensive perspective, the video streams from different cameras are calculated and spliced ​​to generate a panoramic video image or a spliced ​​multi-perspective image to eliminate blind spots, so that the operator can understand the driving environment around the unmanned mine car in time, and make accurate judgments and decisions.

[0033] In some technical solutions, optionally, the unmanned mining vehicle remote control driving method further includes: drawing driving auxiliary lines on the video screen.

[0034] Driving assistance lines provide operators with more intuitive driving assistance information, helping them better judge the driving path and surrounding environment of the unmanned mining vehicle, thereby reducing the risk of accidents. With driving assistance lines, operators can make decisions faster and improve driving efficiency.

[0035] In practical applications, the auxiliary lines are segmented and marked with different colors according to the actual distance threshold, thereby building an intuitive and clear visual reference system for the operator, enabling him to more keenly and accurately judge the driving path of the unmanned mine car, as well as the actual spatial relationship between the unmanned mine car and the surrounding environment.

[0036] In some technical solutions, optionally, the unmanned mining vehicle remote control driving method further includes: performing night vision enhancement processing on the video stream, and / or accessing on-site sound.

[0037] Through night vision enhancement processing, operators can clearly see objects and pedestrians around the unmanned mine car at night or in low light conditions, so as to make accurate judgments and decisions. On-site sound access is achieved using low-latency audio and video remote transmission software, which pushes the on-site sound to the video server synchronously with the video stream, and the takeover system receives the audio and video stream from the video server. The operator can hear the sounds around the unmanned mine car in real time on the interface of the remote takeover system, which helps the operator better understand the environment and conditions around the unmanned mine car, such as traffic conditions, pedestrian sounds, emergency situations, etc.

[0038] According to the second aspect of the present application, the present application proposes a remote control driving device for an unmanned mine car, which adopts the remote control driving method for an unmanned mine car proposed in the first aspect of the present application. The remote control driving device for an unmanned mine car includes: a remote takeover module for obtaining a remote control takeover signal; a map module for drawing a three-dimensional simulation map, and mapping the model of the unmanned mine car onto the three-dimensional simulation map, generating driving assistance information based on the surrounding environment information, and drawing it on the three-dimensional simulation map; the driving assistance information includes obstacle information and / or V2V vehicle information; a perception module for obtaining the surrounding environment information, and generating driving assistance information based on the surrounding environment information; the driving assistance information includes obstacle information and / or V2V vehicle information; a path planning module for generating a guide path, and drawing it on the three-dimensional simulation map; a control module for controlling the unmanned mine car to move along the guide path according to control instructions.

[0039] Therefore, the unmanned mining vehicle remote control driving device has all the beneficial effects of any of the above-mentioned technical solutions, which will not be repeated here.

[0040] Additional aspects and advantages of the present application will become apparent in the following description or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0042] Figure 1 A schematic diagram of the process of the remote control driving method of an unmanned mining vehicle in an embodiment of the present application is shown;

[0043] Figure 2 The left side real-time driving interface of the unmanned mining vehicle remote control driving method in the embodiment of the present application is shown;

[0044] Figure 3 The right side real-time driving interface of the unmanned mining vehicle remote control driving method in the embodiment of the present application is shown;

[0045] Figure 4 The schematic diagram of the structure of the remote control driving device of the unmanned mining vehicle in the embodiment of the present application is shown.

[0046] in, Figure 4 The corresponding relationship between the reference numerals and the component names is as follows:

[0047] 20 unmanned mining vehicle remote control driving device; 210 remote takeover module; 220 map module; 230 perception module; 240 path planning module; 250 control module. DETAILED DESCRIPTION

[0048] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.

[0050] Combine the following Figures 1 to 4 , the unmanned mining vehicle remote control driving method and device provided in the embodiment of the present application are described in detail through specific embodiments and their application scenarios.

[0051] According to the first aspect of the present application, Figure 1 As shown, an embodiment of the present application provides an unmanned mining vehicle remote control driving method, which specifically includes:

[0052] S100, in response to the remote control takeover signal, drawing a three-dimensional simulation map;

[0053] S102, mapping the model of the unmanned mining vehicle onto a three-dimensional simulation map;

[0054] S104, generating driving assistance information according to the surrounding environment information and drawing it on a three-dimensional simulation map; the driving assistance information includes obstacle information and / or V2V vehicle information;

[0055] S106, based on the driving assistance information, generating a guidance path according to the current position information and the target position information of the unmanned mining vehicle, and drawing it on a three-dimensional simulation map;

[0056] S108, controlling the unmanned mining vehicle to move along the guide path according to the control instruction.

[0057] Reference Figure 1 , Figure 2 and Figure 3In the above embodiment, when the remote takeover system receives the remote control takeover signal, it first needs to draw a three-dimensional simulation map. Then the model of the unmanned mine car is mapped to this three-dimensional simulation map, so that the position of the unmanned mine car in the environment can be intuitively seen; it can be understood that the model of the unmanned mine car can include information such as the shape, size, and orientation of the unmanned mine car. The surrounding environment is perceived in real time by the sensors (such as cameras, radars, etc.) on the unmanned mine car, and driving assistance information is generated based on this information, and this information is drawn on the three-dimensional simulation map so that the operator can intuitively see this information; wherein the driving assistance information includes obstacle information and / or V2V (Vehicle-to-Vehicle, unmanned mine workshop communication technology) unmanned mine car information, so as to accurately reflect the environment around the unmanned mine car. After having the driving assistance information, a guide path from the current position to the target position is generated according to the latitude and longitude of the current position of the unmanned mine car and the latitude and longitude of the target position, and it is drawn on the three-dimensional simulation map so that the operator can intuitively see how the unmanned mine car should travel to improve the takeover efficiency. Finally, the unmanned mine car is controlled to drive safely along the guide path according to the control instructions generated by the operator. The control instructions are generated by the operator based on the guide path and driving assistance information, including steering, acceleration, deceleration, parking, etc. This remote control takeover method not only improves the driving experience, but also enhances the safety and reliability of the unmanned mine car.

[0058] In some embodiments, in response to a remote control takeover signal, a three-dimensional simulated map is drawn, specifically including: in response to the remote control takeover signal, obtaining the latest map version number, and performing information verification with the local map version information; when the latest map version number and the local map version information are inconsistent, downloading the latest map file and saving it locally, and performing information verification again; when the latest map version number and the local map version information are consistent, drawing a three-dimensional simulated map based on the local map.

[0059] In the above embodiment, when the remote takeover system receives the remote control takeover signal, it will obtain the latest map version number from the server or cloud, and compare the latest map version number obtained with the locally stored map version information to check whether there is a difference. If the latest map version number and the local map version information are inconsistent, it means that the local map needs to be updated. The remote takeover system will download the latest map file from a reliable source such as a server or cloud and save it locally for subsequent use; after the download is complete, the information is verified again to ensure that the downloaded map file is complete and correct. If the latest map version number and the local map version information are consistent, it means that the local map is already the latest version and does not need to be updated. Based on the locally stored map data, a three-dimensional simulation map is drawn to provide the operator with an intuitive driving environment view, which helps the operator better understand and control the driving environment of the unmanned mine car. By obtaining the latest map version number from the server or cloud, and performing version comparison and updating in real time, the real-time and accuracy of the map data used by the unmanned mine car is ensured.

[0060] In some embodiments, drawing a three-dimensional simulated map based on a local map specifically includes: parsing a local map file and extracting two-dimensional vector map data; converting the two-dimensional vector map data into data in three-dimensional space through coordinate projection conversion and vector data 3D projection, and drawing it on a real-time driving interface.

[0061] In the above embodiment, the local map file usually contains road geometry information (such as road centerline, lane boundary, etc.), traffic signs and signals, topography and other data. By parsing the local map file, various two-dimensional vector data, such as road segments, polygons (such as building outlines), points (such as traffic sign locations), etc., are extracted as the basis for constructing a three-dimensional simulation map. The process of coordinate projection conversion is to convert the coordinates in the geographic coordinate system into three-dimensional space coordinates, which usually involves mathematical operations of map projection and coordinate transformation (such as translation, rotation, scaling, etc.). After completing the coordinate projection conversion, the two-dimensional vector map data is three-dimensionally projected through vector data 3D projection, that is, it is converted into data in three-dimensional space, so that a realistic three-dimensional simulation map can be generated. Finally, the constructed three-dimensional simulation map is drawn on the real-time driving interface. The operator can intuitively see the driving environment around the unmanned mine car through this interface. Through precise coordinate projection conversion and three-dimensional modeling, the system can generate a realistic three-dimensional simulation map, providing the operator with an accurate view of the driving environment, thereby helping to improve the safety and accuracy of remote control takeover of unmanned mine cars.

[0062] In some embodiments, the obstacle information includes obstacle type and / or longitude and latitude; the V2V vehicle information includes at least one of unmanned mining vehicle type, longitude and latitude, and heading angle.

[0063] Obstacle type refers to the specific type of obstacle, such as pedestrians, other unmanned mining vehicles, animals, static obstacles (such as roadblocks, construction areas, etc.). Different types of obstacles will have different effects, so identifying obstacle types is crucial for remote control driving decision-making and planning. Longitude and latitude are the location information of obstacles in the geographic coordinate system. Through longitude and latitude information, the specific location of obstacles can be determined to avoid collisions with them.

[0064] In the above embodiment, the operator can make driving decisions such as acceleration, deceleration, steering, etc. according to obstacle information and V2V vehicle information to ensure driving safety and comfort and avoid collisions with other unmanned mining vehicles and obstacles.

[0065] In some embodiments, when the driving assistance information includes obstacle information, the driving assistance information is generated based on the surrounding environment information and drawn on a three-dimensional simulation map, specifically including: displaying all obstacles within a certain radius with the current position of the unmanned mining vehicle as the center of the circle; wherein different icons are displayed for different obstacle types; dividing the obstacle display area into multiple display areas based on the current position of the unmanned mining vehicle and a preset distance; obstacles located in the same display area have the same display color, and obstacles located in different display areas have different display colors.

[0066] In the above embodiment, the current position of the unmanned mine car is first determined, and then a circular area is drawn with this position as the center. The radius of this circular area can be set as needed, such as 60 meters, to cover obstacles within a certain range around the unmanned mine car. In order to distinguish different types of obstacles, different icons are used to represent them. For example, pedestrians can be represented by a walking man icon, bicycles can be represented by a bicycle icon, and other unmanned mine cars can be represented by different vehicle icons. This allows the operator to quickly identify the type of obstacle, improve the operator's perception of the surrounding environment, and reduce risks. Then, according to the current position of the unmanned mine car and the preset distance threshold, the circular obstacle display area is divided into multiple concentric rings or fan-shaped areas. Each area represents a different distance range, such as close distance, medium distance, and long distance, and different colors are used to represent obstacles located in different display areas. For example, close obstacles can be represented by red, medium distance obstacles can be represented by yellow, and long distance obstacles can be represented by green, etc. In this way, the operator can quickly judge the distance information of the obstacle by color.

[0067] In this embodiment, the braking distance under heavy load is calculated to be 8 meters based on the maximum speed of remote control driving of 15 km / h. The unmanned mining car is designed to display red within a radius of 8 meters (short distance), yellow within 8 meters to 20 meters (medium distance), and green between 20 meters and 60 meters (long distance).

[0068] In some embodiments, the unmanned mine vehicle remote control driving method further includes: when it is determined that the distance between the current position of the unmanned mine vehicle and the obstacle meets a preset alarm threshold condition, issuing an alarm reminder.

[0069] In actual applications, the specific method is to synchronously access the sound. When an obstacle is within the driving route of the unmanned mining car, a "beep" sound prompt is given. If the distance to the obstacle is 10-15 meters, the prompt audio frequency is 1Hz; if the distance to the obstacle is 5-10 meters, the prompt audio frequency is 2Hz; if the distance to the obstacle is less than 5 meters, the prompt sound is long.

[0070] By providing real-time obstacle distance information, it helps operators avoid obstacles in time and reduces the risk of accidents. At the same time, operators can quickly judge the distance of obstacles based on sound prompts and frequency changes, so as to make accurate driving decisions and improve driving efficiency.

[0071] In some embodiments, the unmanned mine car remote control driving method further includes: generating safety fault information according to the unmanned mine car status information; and displaying the safety fault information to the user or operator.

[0072] In the above embodiment, the status information of the unmanned mining vehicle is continuously monitored, including but not limited to slipping, parking slipping, parking abnormality, anti-collision parking, braking abnormality, skidding, sudden position change parking, unchanged position parking, braking force attenuation and thermal decay failure. Once any abnormal fault or safety risk is detected, the corresponding safety fault information will be immediately generated, and the corresponding parking strategy will be triggered, and the safety fault information will be reported to the remote takeover system and displayed to the user or operator.

[0073] In some embodiments, during the takeover process, the current security protection status is reported to the takeover system; the security protection status includes two states: off and on. The takeover system provides a "security protection" button. After turning off security protection, you can turn on security protection again by clicking the button.

[0074] In actual applications, the remote takeover system will provide a "clear fault" button to clear the current safety fault and release the parking strategy, so as to meet the temporary escape needs of the unmanned mining car; the button is clicked to trigger, and there will be a countdown progress bar prompt when triggered. If clicked within 6 seconds, the fault will be cleared for a short time. If it exceeds 6 seconds, the safety protection will be turned off.

[0075] In the above embodiment, by continuously monitoring the status information of the unmanned mine car and handling safety faults in a timely manner, the risk of the unmanned mine car driving process can be significantly reduced. The management function of fault clearing and safety protection status provides users or operators with more control and flexibility, enabling appropriate measures to be taken in emergency situations, further improving the safety and reliability of remote control driving of unmanned mine cars.

[0076] In some embodiments, the remote control driving method of the unmanned mining vehicle further includes: acquiring a video stream of the surrounding environment of the unmanned mining vehicle in real time; and calculating and splicing the video stream to generate a video image.

[0077] In the above embodiment, based on the camera of the unmanned mining vehicle, the video images around the unmanned mining vehicle are captured in real time, and these video streams are uploaded to the remote takeover system. In order to provide a more comprehensive perspective, the video streams from different cameras are calculated and spliced ​​to generate a panoramic video image or a spliced ​​multi-view image to eliminate blind spots, so that the operator can understand the driving environment around the unmanned mining vehicle in a timely manner, so as to make accurate judgments and decisions. For example, video splicing technology is used to achieve splicing frame by frame and push the spliced ​​images directly to the video server of the remote takeover system.

[0078] It is understandable that the visual blind spots can be eliminated by replacing the wide-angle camera and calibrating the distorted image.

[0079] In some embodiments, the unmanned mining vehicle remote control driving method further includes: drawing driving auxiliary lines on the video screen.

[0080] The auxiliary lines use auxiliary lines drawing technology, low-latency overlay drawing method and auxiliary line generation algorithm, combined with the characteristics of the unmanned mining vehicle body to truly fit the body steering direction, and overlay display on the video screen of the remote takeover system to assist in sensing wheel width and turning. The auxiliary lines provide operators with more intuitive driving assistance information, helping them to better judge the driving path and surrounding environment of the unmanned mining vehicle, thereby reducing the risk of accidents. With the auxiliary lines, operators can make decisions faster and improve driving efficiency.

[0081] In practical applications, the auxiliary lines are segmented and marked with different colors according to the actual distance threshold, thereby building an intuitive and clear visual reference system for the operator, enabling him to more keenly and accurately judge the driving path of the unmanned mine car, as well as the actual spatial relationship between the unmanned mine car and the surrounding environment.

[0082] In some embodiments, the unmanned mining vehicle remote control driving method further includes: performing night vision enhancement processing on the video stream; and / or; accessing on-site sound.

[0083] In the above embodiment, at night or in low light conditions, the video stream captured by the unmanned mine car will be processed by night vision enhancement. In terms of hardware, by adding a fill light solution, the exposure around the camera is improved to solve the problem of dark images; in terms of software, based on image processing technology, the takeover system performs night vision enhancement processing on the video stream and displays it. Through night vision enhancement processing, the operator can clearly see the objects and pedestrians around the unmanned mine car at night or in low light conditions, so as to make accurate judgments and decisions. The on-site sound access is implemented by low-latency audio and video remote transmission software, and the on-site sound is pushed to the video server synchronously with the video stream, and the takeover system receives the audio and video stream from the video server. The operator can hear the sounds around the unmanned mine car in real time on the interface of the remote takeover system, which helps the operator better understand the environment and conditions around the unmanned mine car, such as traffic conditions, pedestrian sounds, emergency situations, etc.

[0084] Reference Figure 4 According to the second aspect of the present application, in some embodiments, the present application further provides an unmanned mine car remote control driving device 20, which adopts the unmanned mine car remote control driving method provided by any of the above embodiments, including: a remote takeover module 210, used to obtain a remote control takeover signal; a map module 220, used to draw a three-dimensional simulation map, and map the model of the unmanned mine car to the three-dimensional simulation map, generate driving assistance information according to the surrounding environment information, and draw it on the three-dimensional simulation map; the driving assistance information includes obstacle information and / or V2V vehicle information; a perception module 230, used to obtain the surrounding environment information, and generate driving assistance information according to the surrounding environment information, and draw it on the three-dimensional simulation map; the driving assistance information includes obstacle information and / or V2V vehicle information; a path planning module 240, used to generate a guide path, and draw it on the three-dimensional simulation map; a control module 250, used to control the unmanned mine car to move along the guide path according to the control instruction.

[0085] Since the unmanned mine car remote control driving device 20 utilizes the unmanned mine car remote control driving method provided by any of the above embodiments, it has all the beneficial effects of any of the above embodiments, which will not be described in detail here.

[0086] It should be clarified that in the claims, specification and drawings of the present application, the term "multiple" refers to two or more than two. Unless otherwise clearly defined, the orientation or position relationship indicated by the terms "upper" and "lower" is based on the orientation or position relationship shown in the drawings, which is only for the purpose of more conveniently describing the present application and making the description process easier, rather than indicating or implying that the device or element referred to must have the specific orientation described, be constructed and operated in a specific orientation, so these descriptions cannot be understood as limitations on the present application; the terms "connect", "install", "fix" and the like should be understood in a broad sense. For example, "connection" can be a fixed connection between multiple objects, or a detachable connection between multiple objects, or an integral connection; it can be a direct connection between multiple objects, or an indirect connection between multiple objects through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood based on the specific circumstances of the above data.

[0087] In the claims, specification and drawings of the present application, the description of the terms "one embodiment", "some embodiments", "specific embodiments" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the claims, specification and drawings of the present application, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0088] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for remotely controlling an unmanned mining vehicle, characterized in that: include: In response to the remote control takeover signal, draw a three-dimensional simulation map; Mapping the model of the unmanned mining vehicle onto the three-dimensional simulation map; Generate driving assistance information according to the surrounding environment information, and draw it on the three-dimensional simulation map; The driving assistance information includes obstacle information and / or V2V vehicle information; Based on the driving assistance information, a guidance path is generated according to the current position information and the target position information of the unmanned mining vehicle, and is drawn on the three-dimensional simulation map; The unmanned mining vehicle is controlled to move along the guide path according to the control instruction.

2. The unmanned mining vehicle remote control driving method according to claim 1, characterized in that: The step of drawing a three-dimensional simulation map in response to the remote control takeover signal specifically includes: In response to the remote control takeover signal, the latest map version number is obtained, and the information is verified with the local map version information; If the latest map version number is inconsistent with the local map version information, download the latest map file and save it locally, and then verify the information again; When the latest map version number and the local map version information are consistent, a three-dimensional simulation map is drawn based on the local map.

3. The unmanned mining vehicle remote control driving method according to claim 2, characterized in that: Drawing a three-dimensional simulation map based on a local map specifically includes: Parse local map files and extract 2D vector map data; Through coordinate projection conversion and vector data 3D projection, two-dimensional vector map data is converted into data in three-dimensional space and drawn on the real-time driving interface.

4. The unmanned mining vehicle remote control driving method according to claim 1, characterized in that: The obstacle information includes obstacle type and / or longitude and latitude; The V2V vehicle information includes at least one of the type of unmanned mining vehicle, longitude and latitude, and heading angle.

5. The unmanned mining vehicle remote control driving method according to claim 4, characterized in that: In the case where the driving assistance information includes the obstacle information, generating the driving assistance information according to the surrounding environment information and drawing it on the three-dimensional simulation map specifically includes: Taking the current position of the unmanned mining vehicle as the center of the circle, all obstacles within a preset radius are displayed; different icons are displayed for different types of obstacles; According to the current position of the unmanned mining vehicle and the preset distance, the obstacle display area is divided into multiple display areas; obstacles located in the same display area have the same display color, and obstacles located in different display areas have different display colors.

6. The unmanned mining vehicle remote control driving method according to any one of claims 1 to 5, characterized in that: The unmanned mining vehicle remote control driving method further comprises: Generate safety fault information according to the status information of the unmanned mining vehicle; The safety fault information is displayed to the user or operator.

7. The remote control driving method for an unmanned mining vehicle according to any one of claims 1 to 5, characterized in that: The unmanned mining vehicle remote control driving method further comprises: When the distance between the current position of the unmanned mining vehicle and the obstacle meets the preset alarm threshold condition, an alarm reminder is issued.

8. The remote control driving method for an unmanned mining vehicle according to any one of claims 1 to 5, characterized in that: The unmanned mining vehicle remote control driving method further comprises: Acquire a video stream of the surrounding environment of the unmanned mining vehicle in real time; The video streams are calculated and spliced ​​to generate video images.

9. The unmanned mining vehicle remote control driving method according to claim 8, characterized in that: The unmanned mining vehicle remote control driving method further comprises: Drawing driving assistance lines on the video screen; and / or Performing night vision enhancement processing on the video stream; and / or Access live sound.

10. An unmanned mining car remote control driving device, characterized in that: The unmanned mine car remote control driving method according to any one of claims 1 to 9 is adopted, and the unmanned mine car remote control driving device comprises: Remote takeover module, used to obtain remote control takeover signal; A map module is used to draw a three-dimensional simulation map, map the model of the unmanned mining vehicle onto the three-dimensional simulation map, generate driving assistance information according to the surrounding environment information, and draw it on the three-dimensional simulation map; the driving assistance information includes obstacle information and / or V2V vehicle information; A perception module, used to obtain surrounding environment information and generate driving assistance information based on the surrounding environment information; the driving assistance information includes obstacle information and / or V2V vehicle information; A path planning module, used to generate a guidance path and draw it on the three-dimensional simulation map; A control module is used to control the unmanned mining vehicle to move along the guide path according to a control instruction.

Images