A remote driving method and system for autonomous mining vehicle

Abstract

The present invention relates to a remote driving method and system for an autonomous driving mining vehicle, and relates to the field of autonomous driving technology. The method comprises: collecting environmental data of the mining area through a camera system installed on a vehicle in the mining area, and processing the environmental data to obtain a corresponding environmental perception evaluation value; based on the environmental perception data, detecting and classifying targets in the mining area, and calculating the confidence of each target; based on the detection results of the targets and the mining map data, calculating the optimal driving path of the vehicle; transmitting the driving data and environmental perception data of the vehicle on the optimal driving path to a remote control center in real time, so that the remote driver can monitor the vehicle status in real time through a virtual reality system, and perform manual intervention based on real-time control signals in an emergency; when it is detected that the vehicle is in an abnormal state, triggering an emergency braking operation on the vehicle with an emergency braking force. The present invention can improve the automatic driving accuracy of mining vehicles and the safety of mining operations.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a remote driving method, system, electronic device and non-transitory computer-readable storage medium for an autonomous mining vehicle. Background Art

[0002] Manually driven or semi-automatic driving vehicles are commonly used in mining operations. The driver relies on cameras, radars and other equipment to obtain environmental information and controls the vehicle through manual operations.

[0003] However, the environmental perception system is not very accurate in adverse conditions such as low light and smoke, and cannot accurately identify obstacles. Path planning lacks the ability to respond to dynamic environments in real time and is slow to respond to emergencies. The emergency braking system fails to fully consider relative speed and environmental uncertainty, resulting in unsatisfactory braking effects and possible safety accidents. Summary of the invention

[0004] In response to the technical problems existing in the prior art, the present invention provides a remote driving method, system, electronic device and non-transitory computer-readable storage medium for an autonomous driving mining vehicle, which can improve the autonomous driving accuracy of mining vehicles and the safety of mining operations.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] Collect environmental data of the mining area through a camera system installed on a vehicle in the mining area, and process the environmental data to obtain a corresponding environmental perception evaluation value;

[0007] Based on the environmental perception data, the targets in the mining area are detected and classified, and the confidence of each target is calculated;

[0008] Calculating the optimal driving path of the vehicle based on the detection result of the target and the mining area map data;

[0009] Transmitting the driving data and environmental perception data of the vehicle on the optimal driving path to the remote control center in real time, so that the remote driver can monitor the vehicle status in real time through the virtual reality system and perform manual intervention based on the real-time control signal in an emergency;

[0010] When it is detected that the vehicle is in an abnormal state, an emergency braking operation on the vehicle with an emergency braking force is triggered.

[0011] Optionally, the processing the environmental data to obtain a corresponding environmental perception evaluation value includes:

[0012] Obtaining the distance between each camera in the camera system and the obstacle, and the current speed of the vehicle recorded by each camera;

[0013] Obtaining the weight and image clarity parameters corresponding to each of the cameras;

[0014] The environmental perception evaluation value is determined according to the distance between each camera and the obstacle, the recorded current speed of the vehicle, the corresponding weight and the image clarity parameter.

[0015] Optionally, the environmental perception evaluation value is expressed as:

[0016] ;

[0017] Where E is the environmental perception evaluation value, n is the number of cameras in the camera system, is the weight of the i-th camera, is the image clarity parameter of the i-th camera, is the i-th camera, is the current speed of the vehicle recorded by the i-th camera.

[0018] Optionally, the detecting and classifying the targets in the mining area based on the environmental perception data and calculating the confidence of each target includes:

[0019] Extracting a feature vector of each of the targets and a relative position of each of the targets relative to the vehicle from the environmental perception data;

[0020] Acquire a first adjustment parameter and a second adjustment parameter for calculating the confidence level;

[0021] The confidence level of each target is determined according to the feature vector of each target, the relative position of each target with respect to the vehicle, the first adjustment parameter and the second adjustment parameter.

[0022] Optionally, the confidence of each target is expressed as:

[0023] ;

[0024] in, is the confidence of each target, is the feature vector of the jth target, is the relative position of the jth target relative to the vehicle, , are the first adjustment parameter and the second adjustment parameter respectively.

[0025] Optionally, the calculating the optimal driving path of the vehicle based on the detection result of the target and the mining area map data includes:

[0026] Extracting nodes of multiple paths of the mining area and the path length corresponding to each node from the mining area map data;

[0027] Determining the acceleration and expected speed of the vehicle at each of the nodes according to the detection results;

[0028] Constructing a calculation function of the optimal driving path according to the path length corresponding to each node, the acceleration and expected speed of the vehicle at each node, and combining the first weight and the second weight;

[0029] The optimal driving path is determined according to the calculation function of the optimal driving path.

[0030] Optionally, the calculation function of the optimal driving path is expressed as:

[0031] ;

[0032] in, is the calculation function of the optimal driving path, m is the number of path nodes, is the length of the road segment at node k, is the expected speed of the vehicle at node k, is the acceleration of the vehicle at node k, and are the first weight and the second weight respectively.

[0033] Optionally, the targets include pedestrians, other vehicles and obstacles;

[0034] The camera system includes various combinations of lidar, infrared cameras, 360-degree panoramic cameras, front-view cameras, side-view cameras and rear-view cameras.

[0035] Optionally, the method further includes:

[0036] Determining, based on a detection result of the target, a type of a first obstacle that exists and a position of the first obstacle in a path;

[0037] Determining multiple obstacle avoidance strategies according to the type and location of the first obstacle;

[0038] Under the condition of minimizing the driving time and energy consumption of the vehicle, an optimal obstacle avoidance strategy is determined from the multiple obstacle avoidance strategies.

[0039] The present invention also provides a remote driving system for an autonomous mining vehicle, the system comprising:

[0040] An environmental perception module, used to collect environmental data of the mining area through a camera system installed on a vehicle in the mining area, and process the environmental data to obtain a corresponding environmental perception evaluation value;

[0041] A target detection module, used to detect and classify targets in the mining area based on environmental perception data, and calculate the confidence of each target;

[0042] A path determination module, used to calculate the optimal driving path of the vehicle based on the detection result of the target and the mining area map data;

[0043] A vehicle control module, used for transmitting the driving data and environmental perception data of the vehicle on the optimal driving path to a remote control center in real time, so that the remote driver can monitor the vehicle status in real time through a virtual reality system and perform manual intervention based on real-time control signals in an emergency;

[0044] The emergency braking module is used to trigger an emergency braking operation on the vehicle with an emergency braking force when it is detected that the vehicle is in an abnormal state.

[0045] In addition, to achieve the above-mentioned purpose, the present invention also proposes an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby realizing a remote driving method for an autonomous driving mining vehicle as described above.

[0046] In addition, to achieve the above-mentioned purpose, the present invention also proposes a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, it implements a remote driving method for an autonomous driving mining vehicle as described above.

[0047] The beneficial effects of the present invention are:

[0048] (1) The present invention achieves comprehensive perception of the complex environment of the mining area by installing multiple high-precision cameras (including 360-degree panoramic cameras, front-view cameras, side-view cameras and rear-view cameras) on mining vehicles, combined with laser radar and infrared cameras and other equipment. It can obtain environmental information around the vehicle in real time and accurately, ensuring that the vehicle can effectively identify obstacles, personnel and other vehicles under different lighting, weather and terrain conditions. This efficient environmental perception capability significantly improves the perception accuracy of the autonomous driving system and reduces the driving risks caused by insufficient environmental perception.

[0049] (2) The present invention fully considers the comprehensive influence of driving path, speed and acceleration. By adjusting the dynamic weight parameters, it can balance the energy consumption and driving stability of the vehicle while ensuring the shortest driving time. This optimized path planning method can find the optimal driving path in a complex mining environment, avoid vehicle accidents on dangerous terrain such as steep slopes and curves, and improve the overall efficiency and safety of mining operations.

[0050] (3) The present invention detects and classifies targets in a mining environment in real time, and can accurately distinguish targets such as obstacles, pedestrians, and vehicles, and determine their relative positions and driving status. The algorithm can maintain efficient target recognition and classification capabilities in complex mining environments, and can accurately identify potential dangers even in conditions of poor lighting or limited vision. This greatly reduces the safety hazards caused by inaccurate target detection and improves the vehicle's autonomous driving capabilities and driving safety.

[0051] In summary, the present invention realizes the efficient, accurate and intelligent operation of autonomous driving mining vehicles in complex environments by optimizing environmental perception, path planning, target detection and safety control mechanisms. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a remote driving method for an autonomous mining vehicle provided by the present invention;

[0053] Figure 2 A schematic diagram of the structure of a remote driving system for an autonomous mining vehicle provided by the present invention;

[0054] Figure 3 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0055] Figure 4 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0057] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0058] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0059] See also Figure 1 , provides a flow chart of a remote driving method of an autonomous mining vehicle of the present invention, comprising the following steps:

[0060] Step 201: Collect environmental data of the mining area through a camera system installed on a vehicle in the mining area, and process the environmental data to obtain a corresponding environmental perception evaluation value.

[0061] Among them, the camera system can include various combinations of lidar, infrared camera, 360-degree panoramic camera, front-view camera, side-view camera and rear-view camera.

[0062] In some embodiments, step 201 may include:

[0063] Obtaining the distance between each camera in the camera system and the obstacle, and the current speed of the vehicle recorded by each camera;

[0064] Obtaining the weight and image clarity parameters corresponding to each of the cameras;

[0065] The environmental perception evaluation value is determined according to the distance between each camera and the obstacle, the recorded current speed of the vehicle, the corresponding weight and the image clarity parameter.

[0066] In some embodiments, the environmental perception assessment value is expressed as:

[0067] ;

[0068] Where E is the environmental perception evaluation value, n is the number of cameras in the camera system, is the weight of the i-th camera, is the image clarity parameter of the i-th camera, is the i-th camera, is the current speed of the vehicle recorded by the i-th camera.

[0069] In the specific implementation, E is the environmental perception evaluation value, which is used to quantify the overall perception ability of the vehicle in the current mining environment. The higher the value, the stronger the system's perception of the environment, and the clearer and more comprehensive the environmental information.

[0070] n is the number of cameras in the camera system. The more cameras there are, the more comprehensive the environmental data collected will be, which can cover a wider field of view and reduce environmental blind spots.

[0071] is the weight of the ith camera, which is used to indicate the importance of different cameras in environmental perception. The weight value can be set according to factors such as camera installation location, field of view, and importance.

[0072] is the image clarity parameter of the i-th camera, which is used to quantify the quality of the camera image. Image clarity may be affected by the camera's own resolution, light conditions, weather conditions, etc. For example, in a day with good light, the clarity is high. The value of is larger; in the dark night or foggy day, the clarity is lower. The value of is smaller.

[0073] is the distance between the ith camera and the obstacle, usually in meters (m). The closer the distance, the greater the impact of the obstacle on the vehicle's driving, and the higher the importance of environmental perception. This parameter can be extracted from the camera image by lidar, ultrasonic sensor or computer vision algorithm.

[0074] is the current speed of the vehicle recorded by the ith camera, usually in meters per second (m / s). The higher the speed, the higher the real-time requirements of the environment perception system for environmental information. For example, when the vehicle is driving at high speed, the dynamic changes of obstacles will be faster, requiring the camera system to recognize and process environmental information faster.

[0075] Specifically, Partially represents the perception ability of the i-th camera to the environment. When it is larger, it means that the camera's image clarity is high and the environmental information is reliable. Indicates the ratio of obstacle distance to vehicle speed. Distance above obstacles The importance of environmental perception increases.

[0076] when Larger or When it is smaller, the ratio increases, indicating that the vehicle has more time to respond to changes in the environment ahead, so the environmental perception pressure is lower; conversely, when Smaller or When it is larger, the ratio decreases, indicating that the vehicle needs to react faster and cope with higher requirements for environmental changes.

[0077] Weights of different cameras Used to adjust the contribution of each camera in the comprehensive environment perception. For example, the front-view camera is used to monitor the road ahead, so a higher weight may be set; the side-view camera monitors the surrounding environment, so a lower weight may be set.

[0078] After calculating the E value of each camera, the E values ​​of the same camera system can be added together, and the final environmental perception evaluation value is 2.91, which means that the current camera system has a good overall perception ability of the mining environment. The vehicle can judge the safety and passability of the current environment based on this value and make corresponding adjustments in driving control.

[0079] Step 202: Detect and classify targets in the mining area based on the environmental perception data, and calculate the confidence level of each target.

[0080] Targets may include pedestrians, other vehicles, and obstacles.

[0081] In some embodiments, step 202 may include:

[0082] Extracting a feature vector of each of the targets and a relative position of each of the targets relative to the vehicle from the environmental perception data;

[0083] Acquire a first adjustment parameter and a second adjustment parameter for calculating the confidence level;

[0084] The confidence level of each target is determined according to the feature vector of each target, the relative position of each target with respect to the vehicle, the first adjustment parameter and the second adjustment parameter.

[0085] In some embodiments, the confidence level of each of the targets is expressed as:

[0086] ;

[0087] in, is the confidence of each target, is the feature vector of the jth target, is the relative position of the jth target relative to the vehicle, , are the first adjustment parameter and the second adjustment parameter respectively.

[0088] In the specific implementation, is the confidence of the target, and its value range is [0,1]. The closer the confidence is to 1, the more confident the system is in identifying the target; the closer the confidence is to 0, the higher the uncertainty of the system in identifying the target.

[0089] Confidence is used to judge the vehicle's decisions in actual operation, such as whether to avoid obstacles or slow down.

[0090] is the feature vector of the jth target, which is usually composed of multi-dimensional feature data, such as the target's shape, color, size, edge information, texture features, etc. For autonomous mining vehicles, the feature vector Including the shape of the target (such as workers, obstacles, other vehicles, etc.), dynamic characteristics (such as the target's movement direction and speed), etc.

[0091] is the relative position of the jth target relative to the vehicle. It can be the relative coordinates of the target (such as distance, angle, etc.) or a multidimensional vector that describes the relative position of the target in three-dimensional space. This parameter is crucial for judging the relative safety of the target and the vehicle. For example, if the target is directly in front of the vehicle and the distance is close, the system will give a higher degree of attention, thereby increasing the confidence of the target.

[0092] , are the first adjustment parameter and the second adjustment parameter respectively. and relative position The degree of influence on the confidence calculation. As the first adjustment parameter, it is usually used to enlarge or reduce the feature vector Contribution to confidence. For example, when When it is larger, the weight of the feature vector in the confidence judgment increases, and the system relies more on the visual features of the target. As the second adjustment parameter, it is used to adjust the relative position The impact on confidence. When it is larger, the system pays more attention to the spatial position relationship of the target.

[0093] Specifically, the formula for calculating confidence is equivalent to a standard logistic regression model (LogisticRegression), whose output value is limited to [0,1]. The linear combination within the formula After the transformation of the Sigmoid function, a confidence value is obtained .

[0094] The input of the Sigmoid function can be any real number, but the output is limited to (0,1), indicating that the linear combination of the eigenvector and the relative position No matter how large or small the value of is, the confidence output is always within a controllable range.

[0095] when When it is larger (such as a positive number), it means that the target feature is significant and position sensitive, and the confidence Close to 1; when When it is small (such as a negative number), it means that the target feature is not obvious or the position is not sensitive, and the confidence Close to 0.

[0096] Over adjustment The value of can be used to make different confidence trade-offs for different target types. For example, when the vehicle pays more attention to the visual features of the target (such as color, shape, etc.), it can increase When the vehicle is more concerned about the spatial position of the target (such as distance, angle, etc.), it can be increased The value of .

[0097] The autonomous mining vehicle of the present invention can comprehensively judge the importance of the target and the reliability of recognition by combining the visual features and relative position of the target. , The value of can more precisely identify and distinguish different target types, thereby effectively improving the vehicle's autonomous driving capability and safety in complex mining environments.

[0098] Step 203: Calculate the optimal driving path of the vehicle based on the detection result of the target and the mining area map data.

[0099] In some embodiments, step 203 may include:

[0100] Extracting nodes of multiple paths of the mining area and the path length corresponding to each node from the mining area map data;

[0101] Determining the acceleration and expected speed of the vehicle at each of the nodes according to the detection results;

[0102] Constructing a calculation function of the optimal driving path according to the path length corresponding to each node, the acceleration and expected speed of the vehicle at each node, and combining the first weight and the second weight;

[0103] The optimal driving path is determined according to the calculation function of the optimal driving path.

[0104] In some embodiments, the calculation function of the optimal driving path is expressed as:

[0105] ;

[0106] in, is the calculation function of the optimal driving path, m is the number of path nodes, is the length of the road segment at node k, is the expected speed of the vehicle at node k, is the acceleration of the vehicle at node k, and are the first weight and the second weight respectively.

[0107] In the specific implementation, It is the calculation function of the optimal driving path. The goal is to find an optimal path that minimizes the total cost from the starting point to the end point.

[0108] m is the number of path nodes, that is, the number of segments from the starting point to the end point of the vehicle.

[0109] is the length of the road segment at node k, usually in meters (m). This value reflects the distance traveled by the vehicle in this segment.

[0110] is the desired speed of the vehicle at node k, in meters per second (m / s). A higher speed will shorten the travel time, but may also result in increased energy consumption. is the reciprocal of speed, which is used to characterize the inverse relationship between travel time and speed. When the speed is lower, the travel time is longer, and vice versa.

[0111] is the acceleration of the vehicle at node k, in (m / s²). The magnitude of the acceleration directly affects the energy consumption and driving stability of the vehicle.

[0112] and are the first weight and the second weight respectively. The first weight is used to adjust the influence of speed in the objective function. Generally speaking, The larger the value of , the more importance is attached to travel time in path planning. The second weight is used to adjust the influence of acceleration in the objective function. Usually, The larger the value, the more emphasis is placed on optimizing smooth driving and energy consumption.

[0113] As an example, suppose that the calculation function , the cost C1 of node 1 is 101.5, the cost C2 of node 2 is 200.93, and the cost C3 of node 3 is 151.22. When the vehicle's path is node 1 to node 2 to node 3, the total cost C=453.62. Assuming that the total cost of another path is 460, it is obvious that its total cost is larger, and the path from node 1 to node 2 to node 3 can be determined as the current optimal path.

[0114] It should be noted that in actual applications, the mining path may be more complicated, and the changes in the number of nodes m and variables will affect the final result. Dynamic programming or other optimization algorithms can be used to quickly find the optimal path. The above is a simplified example, and more detailed calculations can be performed according to specific scenarios.

[0115] In some embodiments, the present invention may further include:

[0116] Determining, based on a detection result of the target, a type of a first obstacle that exists and a position of the first obstacle in a path;

[0117] Determining multiple obstacle avoidance strategies according to the type and location of the first obstacle;

[0118] Under the condition of minimizing the driving time and energy consumption of the vehicle, an optimal obstacle avoidance strategy is determined from the multiple obstacle avoidance strategies.

[0119] In some embodiments, the system can first scan the front environment in real time through sensors such as on-board cameras and lidars, and detect target obstacles. Through target recognition algorithms (such as deep learning algorithms), the system can classify detected targets into different types, such as static obstacles (such as rocks, equipment), dynamic obstacles (such as pedestrians, vehicles), etc. At the same time, the system determines the specific position of the obstacle in the current driving path based on the position information provided by the sensor, including the distance between the obstacle and the vehicle, the relative direction, and the relative coordinates of the obstacle in the path. This step lays the foundation for the subsequent selection of obstacle avoidance strategies.

[0120] After determining the type and location of the obstacle, the system generates a variety of obstacle avoidance strategies based on the characteristics of different types of obstacles. Specifically: For stationary obstacles, such as rocks or equipment, obstacle avoidance strategies may include detours, deceleration, or emergency stops. For dynamic obstacles, such as pedestrians or other vehicles, the system needs to consider the movement trajectory of the obstacle and develop more flexible obstacle avoidance strategies, such as changing lanes, slowing down and following, or turning in advance. For each strategy, the system considers the obstacle avoidance path, the speed adjustment required for the vehicle, and the required acceleration or deceleration measures, thereby generating multiple alternative obstacle avoidance paths and driving plans.

[0121] In some embodiments, after generating multiple obstacle avoidance strategies, the system will evaluate each strategy, with the goal of selecting an optimal strategy that can minimize driving time and energy consumption while meeting safety requirements. Specific considerations include:

[0122] Travel time: The estimated time for each obstacle avoidance strategy. For example, a detour strategy may increase the travel time, while a speed reduction strategy may result in a longer time.

[0123] Energy consumption: Each obstacle avoidance strategy has a corresponding energy consumption. For example, lane changes, sudden acceleration or sudden braking usually increase energy consumption. The system will comprehensively evaluate these factors and select the optimal strategy through optimization algorithms (such as Bayesian optimization, dynamic programming, etc.) to ensure that the vehicle bypasses obstacles in the most economical way and continues to perform the scheduled driving task.

[0124] The intelligent obstacle avoidance strategy selection method of the present invention effectively improves the driving safety and efficiency of vehicles in complex mining environments. By combining the type and location of obstacles and the vehicle's own status (such as speed, direction, etc.), the system can respond in real time to avoid collisions or traffic interruptions. The selection of the optimal strategy can not only reduce the vehicle's driving time, but also reduce unnecessary energy consumption, extend the vehicle's operating time and endurance, and provide more efficient and safe autonomous driving support for mining operations.

[0125] Step 204: Transmit the driving data and environmental perception data of the vehicle on the optimal driving path to the remote control center in real time, so that the remote driver can monitor the vehicle status in real time through the virtual reality system and perform manual intervention based on the real-time control signal in an emergency.

[0126] In some embodiments, the real-time control signal may be represented as:

[0127] ;

[0128] in, It is a real-time control signal used to adjust the driving state of the vehicle. The size of this signal determines the strength and response of the vehicle to perform a certain operation (such as steering, acceleration or braking, etc.). When it is larger, it means that the vehicle needs stronger adjustment or intervention, such as emergency braking, sharp turning, etc. When it is small, it means that the vehicle only needs minor adjustments, such as fine-tuning direction or speed.

[0129] The human control coefficient is used to control the influence of human intervention on the entire control signal. This coefficient can be adjusted according to the actual situation. When the value is large, it indicates that the influence of human control is stronger, which is suitable for scenarios that require human control. When the value is smaller, it indicates that the system relies more on the autonomous judgment and execution of the autonomous driving system.

[0130] z is the number of intervention operations, that is, how many different types of intervention operations the system needs to consider. These operations may include steering, acceleration, braking, changing the driving route, etc. Each operation will have a different impact on the driving state of the vehicle, and the system needs to comprehensively consider the overall impact of all operations on the vehicle.

[0131] It is The standardized response time of a type of operation is used to measure the time required for the vehicle to perform a certain operation. For example, the response time of a steering operation may be short, while the response time of an emergency braking operation may be relatively long. The standardization of this parameter can convert the response time of different operations into the same dimension, which is convenient for unified comparison and calculation.

[0132] It is the depth of influence of such operation on vehicle behavior, which is usually used to quantify the degree of influence of the operation on the vehicle's driving state, such as the size of the steering angle, the amplitude of acceleration or deceleration, etc. This parameter reflects the intervention intensity of different operations on vehicle behavior. The greater the depth of influence, the more significant the adjustment effect of the operation on the vehicle's driving path, speed or direction.

[0133] It is the vehicle's response sensitivity to such operations, indicating the speed and degree of the vehicle's response to the operation. The higher the response sensitivity, the faster and more obvious the vehicle's execution effect on the operation. For example, when the vehicle's brake system is highly sensitive, a slight braking operation can produce a significant deceleration effect; while a steering operation with a low response sensitivity may require a larger steering signal to produce a significant steering effect.

[0134] It is the external environment uncertainty correction coefficient. It is used to adjust the impact of the external environment (such as weather, road conditions, surrounding obstacles, etc.) on vehicle operation. This coefficient can be adjusted according to the degree of uncertainty of the external environment. For example, when the environment is very complex or dangerous (such as landslides in mining areas, extremely low visibility, etc.), you can increase value, thereby increasing the depth of influence on vehicle operation degree of importance.

[0135] Specifically, Combination means l The overall impact time of the class operation, taking into account the response time of the operation and operating depth under the influence of external environment . Indicates The ultimate impact of such operation on the vehicle. Response sensitivity When it is larger, it means that the vehicle reacts slowly to the operation, and the system needs to amplify the control signal of the operation; conversely, when the response sensitivity is smaller, the system needs to reduce the control signal of the operation. The weighted sum of the effects of all intervention operations is multiplied by the human control coefficient. , used to adjust the overall intensity of manual intervention.

[0136] The present invention can effectively combine the response time, impact depth, environmental factors and vehicle response sensitivity of different intervention operations to give a comprehensive vehicle control signal. In practical applications, various parameters can be dynamically adjusted according to the operating status of the vehicle and the complexity of the external environment to achieve more efficient and safer remote driving control.

[0137] Step 205: When it is detected that the vehicle is in an abnormal state, trigger an emergency braking operation on the vehicle with an emergency braking force.

[0138] In some embodiments, the emergency braking force can be expressed as:

[0139] ;

[0140] in, is the current emergency braking force, with a value range of [0, ], which is used to characterize the strength of the braking force applied by the vehicle in an emergency. near When the braking intensity reaches the maximum value, the vehicle is braked urgently; when When it is close to 0, it means that the vehicle does not need emergency braking or only needs light braking.

[0141] It is the maximum braking force of the braking system, which is usually determined by the physical properties of the braking system. It indicates the maximum braking force that the vehicle can apply in an emergency, in Newtons (N). This value is determined by the type of vehicle and the configuration of the braking system. For example, the maximum braking force of heavy mining vehicles is usually larger, while the maximum braking force of light vehicles is relatively smaller.

[0142] is the minimum safe distance allowed. If the actual distance between the vehicle and the obstacle is If the distance is less than this, the system will determine that there is a risk of collision and emergency braking is required. This parameter is usually set according to the vehicle's driving speed, braking performance and specific conditions in the mining area to ensure that the vehicle can stop in time before a collision occurs.

[0143] is the current distance between the vehicle and the obstacle, which means the current distance between the vehicle and the obstacle at time t, usually in meters (m). It is a dynamically changing parameter that can be measured in real time by lidar, camera or other sensors. Close to or less than When the vehicle is too close to the obstacle, emergency braking is required.

[0144] It is the relative speed between the vehicle and the obstacle, which indicates the relative speed between the vehicle and the obstacle, usually in meters per second (m / s). The relative speed can be positive or negative: a positive value indicates that the vehicle and the obstacle are moving towards each other (such as a head-on collision); a negative value indicates that the vehicle and the obstacle are moving in the same direction, but the obstacle is faster than the vehicle (that is, the vehicle is moving backward relative to the obstacle). The greater the relative speed, the faster the two are approaching, and a stronger braking force is required; the smaller the relative speed, the slower the approach speed or the farther away state, and the braking force can be appropriately reduced.

[0145] q and They are the third adjustment coefficient and the fourth adjustment coefficient, respectively, which are used to adjust the influence of the minimum safety distance and relative speed on the braking signal. q is used to control the current distance Relative to the minimum safety distance When the q value is large, the vehicle is more sensitive to changes in distance, that is, the shorter the distance, the stronger the braking force. To control the relative speed When A larger value means that the system is more sensitive to speed changes, and the braking force increases more significantly when the relative speed is higher.

[0146] This item is used to characterize the effect of the distance between the current vehicle and the obstacle on the braking force. Less than When the vehicle is too close to the obstacle, the item takes a positive value, indicating that the braking force needs to be increased; when Greater than When , it means the vehicle is within a safe distance. If this item takes a negative value, the braking force can be reduced.

[0147] Used to indicate the effect of relative speed on braking force. When it is larger (the vehicle approaches the obstacle quickly), this item takes a positive value, indicating that a stronger braking force is needed; when When the load is small, the braking force can be appropriately reduced.

[0148] When the distance between the vehicle and the obstacle When the relative speed is small, When larger, the internal The term is large, and the input value of the Sigmoid function tends to be positive, resulting in is close to 0, so Close to maximum braking force , that is, emergency braking is required.

[0149] When the vehicle is far away from the obstacle or the relative speed is low, the internal term is negative and the input of the Sigmoid function tends to be negative, resulting in Approaching positive infinity, Close to 0, no braking or only slight braking is required.

[0150] The present invention can dynamically adjust the braking strength of the vehicle by calculating the distance and relative speed between the vehicle and the obstacle in real time. By adjusting the parameters, it can flexibly respond to different emergency situations, thus improving the safety and reliability of the autonomous driving vehicle in the complex environment of the mining area.

[0151] See also Figure 2 , Figure 2 A schematic structural diagram of a remote driving system for an autonomous mining vehicle provided by the present invention.

[0152] like Figure 2 As shown, a remote driving system for an autonomous mining vehicle proposed in an embodiment of the present invention includes:

[0153] The environment perception module 301 is used to collect the environment data of the mining area through a camera system installed on a vehicle in the mining area, and process the environment data to obtain a corresponding environment perception evaluation value;

[0154] The target detection module 302 is used to detect and classify the targets in the mining area based on the environmental perception data, and calculate the confidence of each target;

[0155] A path determination module 303 is used to calculate the optimal driving path of the vehicle based on the detection result of the target and the mining area map data;

[0156] The vehicle control module 304 is used to transmit the driving data and environmental perception data of the vehicle on the optimal driving path to the remote control center in real time, so that the remote driver can monitor the vehicle status in real time through the virtual reality system and perform manual intervention based on the real-time control signal in an emergency;

[0157] The emergency braking module 305 is used to trigger an emergency braking operation on the vehicle with an emergency braking force when it is detected that the vehicle is in an abnormal state.

[0158] See also Figure 3 , Figure 3 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented:

[0159] Collect environmental data of the mining area through a camera system installed on a vehicle in the mining area, and process the environmental data to obtain a corresponding environmental perception evaluation value;

[0160] Based on the environmental perception data, the targets in the mining area are detected and classified, and the confidence of each target is calculated;

[0161] Calculating the optimal driving path of the vehicle based on the detection result of the target and the mining area map data;

[0162] Transmitting the driving data and environmental perception data of the vehicle on the optimal driving path to the remote control center in real time, so that the remote driver can monitor the vehicle status in real time through the virtual reality system and perform manual intervention based on the real-time control signal in an emergency;

[0163] When it is detected that the vehicle is in an abnormal state, an emergency braking operation on the vehicle with an emergency braking force is triggered.

[0164] See also Figure 4 , Figure 4 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 4 As shown, this embodiment provides a computer-readable storage medium 500 on which a computer program 411 is stored.

[0165] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0166] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0168] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0170] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0171] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A remote driving method for an autonomous mining vehicle, characterized in that: The method comprises: The environmental data of the mining area is collected by a camera system installed on a vehicle in the mining area, and the environmental data is processed to obtain a corresponding environmental perception evaluation value; the environmental perception evaluation value is expressed as: ; Where E is the environmental perception evaluation value, n is the number of cameras in the camera system, is the weight of the i-th camera, is the image clarity parameter of the i-th camera, is the i-th camera, is the current speed of the vehicle recorded by the i-th camera; Based on the environmental perception data, the targets in the mining area are detected and classified, and the confidence of each target is calculated; Calculating the optimal driving path of the vehicle based on the detection result of the target and the mining area map data; Transmitting the driving data and environmental perception data of the vehicle on the optimal driving path to the remote control center in real time, so that the remote driver can monitor the vehicle status in real time through the virtual reality system and perform manual intervention based on the real-time control signal in an emergency; When it is detected that the vehicle is in an abnormal state, triggering an emergency braking operation on the vehicle with an emergency braking force; The detecting and classifying the targets in the mining area based on the environmental perception data and calculating the confidence of each target include: Extracting a feature vector of each of the targets and a relative position of each of the targets relative to the vehicle from the environmental perception data; Acquire a first adjustment parameter and a second adjustment parameter for calculating the confidence level; The confidence level of each target is determined according to the feature vector of each target, the relative position of each target with respect to the vehicle, the first adjustment parameter and the second adjustment parameter.

2. The remote driving method of an autonomous mining vehicle according to claim 1, characterized in that: The processing of the environmental data to obtain a corresponding environmental perception evaluation value includes: Obtaining the distance between each camera in the camera system and the obstacle, and the current speed of the vehicle recorded by each camera; Obtaining the weight and image clarity parameters corresponding to each of the cameras; The environmental perception evaluation value is determined according to the distance between each camera and the obstacle, the recorded current speed of the vehicle, the corresponding weight and the image clarity parameter.

3. The remote driving method of an autonomous mining vehicle according to claim 1, characterized in that: The confidence of each of the targets is expressed as: ； in, is the confidence of each target, is the feature vector of the jth target, is the relative position of the jth target relative to the vehicle, are the first adjustment parameter and the second adjustment parameter respectively.

4. The remote driving method of an autonomous mining vehicle according to claim 3, characterized in that: The calculating the optimal driving path of the vehicle based on the detection result of the target and the mining area map data includes: Extracting nodes of multiple paths of the mining area and the path length corresponding to each node from the mining area map data; Determining the acceleration and expected speed of the vehicle at each of the nodes according to the detection results; Constructing a calculation function of the optimal driving path according to the path length corresponding to each node, the acceleration and expected speed of the vehicle at each node, and combining the first weight and the second weight; The optimal driving path is determined according to the calculation function of the optimal driving path.

5. The remote driving method of an autonomous mining vehicle according to claim 4, characterized in that: The calculation function of the optimal driving path is expressed as: ； in, is the calculation function of the optimal driving path, m is the number of path nodes, is the length of the road segment at node k, is the expected speed of the vehicle at node k, is the acceleration of the vehicle at node k, and are the first weight and the second weight respectively.

6. The remote driving method of an autonomous mining vehicle according to claim 5, characterized in that: Said targets include pedestrians, other vehicles, and obstacles; The camera system includes various combinations of lidar, infrared cameras, 360-degree panoramic cameras, front-view cameras, side-view cameras and rear-view cameras.

7. The remote driving method of an autonomous mining vehicle according to any one of claims 1 to 6, characterized in that: The method further comprises: Determining, based on a detection result of the target, a type of a first obstacle that exists and a position of the first obstacle in a path; Determining multiple obstacle avoidance strategies according to the type and location of the first obstacle; Under the condition of minimizing the driving time and energy consumption of the vehicle, an optimal obstacle avoidance strategy is determined from the multiple obstacle avoidance strategies.

8. A remote driving system for an autonomous mining vehicle, characterized in that: The system comprises: The environment perception module is used to collect the environment data of the mining area through a camera system installed on a vehicle in the mining area, and process the environment data to obtain a corresponding environment perception evaluation value; the environment perception evaluation value is expressed as: ; Where E is the environmental perception evaluation value, n is the number of cameras in the camera system, is the weight of the i-th camera, is the image clarity parameter of the i-th camera, is the i-th camera, is the current speed of the vehicle recorded by the i-th camera; A target detection module, used to detect and classify targets in the mining area based on environmental perception data, and calculate the confidence of each target; A path determination module, used to calculate the optimal driving path of the vehicle based on the detection result of the target and the mining area map data; A vehicle control module, used for transmitting the driving data and environmental perception data of the vehicle on the optimal driving path to a remote control center in real time, so that the remote driver can monitor the vehicle status in real time through a virtual reality system and perform manual intervention based on real-time control signals in an emergency; An emergency braking module, configured to trigger an emergency braking operation on the vehicle with an emergency braking force when detecting that the vehicle is in an abnormal state; The detecting and classifying the targets in the mining area based on the environmental perception data and calculating the confidence of each target include: Extracting a feature vector of each of the targets and a relative position of each of the targets relative to the vehicle from the environmental perception data; Acquire a first adjustment parameter and a second adjustment parameter for calculating the confidence level; The confidence level of each target is determined according to the feature vector of each target, the relative position of each target with respect to the vehicle, the first adjustment parameter and the second adjustment parameter.

Images