A 5G-based visual recognition and path planning system for unmanned mining trucks

Abstract

The present invention discloses a visual recognition and path planning system for unmanned mining trucks based on 5G, which relates to the field of communication technology. The system comprises a communication module for realizing real-time data transmission based on the high-bandwidth and low-latency characteristics of the 5G network, an on-board visual sensor module for acquiring real-time information of the surrounding environment, a target detection and recognition module for detecting and identifying targets based on the environmental information acquired by the on-board visual sensor module, a path planning module for performing path planning of the unmanned mining truck using map data updated in real time, and an information synchronization module for eliminating information synchronization errors based on the communication characteristics of the 5G network, so as to ensure accurate operation and navigation of the unmanned mining truck. The visual recognition and path planning system for unmanned mining trucks based on 5G solves the problem in the prior art that the unmanned mining trucks make erroneous responses due to the complex terrain of the mining operation sites, thereby reducing the stability and operation reliability of the entire system.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a 5G-based unmanned mining truck visual recognition and path planning system. Background Art

[0002] In modern mining operations, unmanned mining truck technology has gradually been widely used. Relying on advanced image processing algorithms and path planning technologies, it has realized the automatic control and management of mining trucks in complex environments. Among them, 5G communication technology, with its advantages of high speed, low latency and large-scale connection, provides important technical support for the remote control and real-time data transmission of unmanned mining trucks. Through the 5G network, unmanned mining trucks can collect and analyze large amounts of data in the mining environment in real time, formulate the optimal driving route, effectively improve operational efficiency, and greatly reduce the safety hazards of manual driving.

[0003] However, in the special and complex environment of mines, the application of unmanned driving technology still faces certain challenges. The terrain of mining sites is usually complex, with a large number of obstacles such as rocks, buildings and trees. These obstacles will cause the obstruction of 5G signals, which in turn affects the data transmission and communication synchronization between the unmanned mining truck and the central control system. When signal transmission is blocked, information delays or transmission errors will occur, resulting in the inability of the unmanned mining truck to receive or send instructions in time, seriously affecting the path planning and decision-making process of the vehicle. This information synchronization error problem, especially at critical decision-making moments, causes the unmanned mining truck to make incorrect responses, thereby reducing the stability and operational reliability of the entire system. Therefore, how to ensure the stable transmission of 5G signals under complex terrain and improve the coordination and real-time performance of the unmanned driving system are technical problems that need to be solved urgently in the current unmanned mining truck technology. Summary of the invention

[0004] The purpose of the present invention is to provide a 5G-based unmanned mining truck visual recognition and path planning system to solve the problem in the prior art that due to the complex terrain of the mining workplace, obstacles may cause the obstruction of 5G signals, thereby affecting the data transmission and communication synchronization between the unmanned mining truck and the central control system. When the signal transmission is blocked, information delays or transmission errors will occur, resulting in the unmanned mining truck being unable to receive or send instructions in time, seriously affecting the vehicle's path planning and decision-making process, and information synchronization errors, especially at critical decision-making moments, causing the unmanned mining truck to make incorrect responses, thereby reducing the stability of the entire system and the reliability of operations.

[0005] To achieve the above objectives, the present invention provides the following technical solutions: a 5G-based unmanned mining truck visual recognition and path planning system, the system comprising:

[0006] Communication module, used to achieve real-time data transmission based on the high bandwidth and low latency characteristics of 5G network;

[0007] On-board visual sensor module, used to obtain real-time information about the surrounding environment;

[0008] The target detection and recognition module is used to detect and recognize targets based on the environmental information obtained by the on-board visual sensor module;

[0009] The path planning module is used to plan the path of the unmanned mining truck using real-time updated map data;

[0010] Information synchronization module, which is used to eliminate information synchronization errors based on the communication characteristics of the 5G network to ensure accurate operation and navigation of unmanned mining trucks;

[0011] The target detection and recognition module detects and recognizes the target based on the environmental information obtained by the vehicle-mounted visual sensor module, including: real-time positioning through convolutional neural network and YOLO algorithm, and the specific formula is:

[0012] × ,

[0013] in, represents the feature map, represents the convolution kernel, represents the input image;

[0014] The cost function in path planning is calculated based on the distance to the surrounding targets and the weight of the target importance, specifically:

[0015] ,

[0016] in, represents the cost function of path planning, Represents the displacement cost of the current target, Indicates the number of targets involved, represents the precision weight of the target operation, represents the importance weight of the target, Indicates the distance to surrounding targets. Indicates the position of the surrounding targets on the horizontal axis, Indicates the position of surrounding targets on the ordinate;

[0017] The path planning module uses the map data updated in real time to plan the path of the unmanned mining truck, including: minimizing the total time by going to each target time, and calculating the objective function of the path planning in combination with dynamic information, specifically:

[0018] ,

[0019] in, represents the objective function of path planning, Indicates the number of targets involved, Indicates to The time of the target, Indicates The time from one goal to the next goal;

[0020] According to the impact of the environmental information of the surrounding targets on the path, the time to the target is adjusted in real time, specifically:

[0021] ,

[0022] in, Indicates that two targets are at a specific time The actual time elapsed, Indicates time, Indicates the time to reach the target under ideal conditions. Indicates the delay time caused by environmental factors.

[0023] Preferably, the communication module realizes real-time data transmission based on the high bandwidth and low latency characteristics of the 5G network, including:

[0024] Real-time acquisition of signal blocking areas caused by changes in the mine's geographical environment;

[0025] Adjust the 5G base station power according to the size of the obstructed area to ensure unobstructed communications in key areas;

[0026] Formulate a multi-hop communication link backup strategy based on vehicle position and speed parameters;

[0027] The dynamic data switching process is started when it is detected that the signal strength at a certain point on the communication link drops by more than the set threshold H, where H represents the set signal strength threshold. When the signal strength drops below A, it indicates that the signal is severely blocked.

[0028] Preferably, the formulating of a multi-hop communication link backup strategy includes:

[0029] Monitor the 5G signal strength and its changes around each vehicle;

[0030] Calculate the communication range of all communicable nodes around each vehicle;

[0031] Form a backup route based on the neighboring nodes with the best signal within the communication range;

[0032] It is determined whether the vehicle enters the predefined critical shielding area R. If it enters the area R, the link is triggered to automatically switch to the best backup route to ensure continuous data transmission, where the area R represents a fixed location prone to communication problems.

[0033] Preferably, the vehicle-mounted visual sensor module acquires real-time information of the surrounding environment including acquiring images and depth information around the mining truck through a camera and a laser radar.

[0034] Preferably, the communication module realizes real-time data transmission based on the high bandwidth and low latency characteristics of the 5G network and further includes:

[0035] Collect large amounts of information generated by multiple on-board sensors I;

[0036] Evaluate the transmission delay T5 to ensure that critical instructions can be delivered to the destination before the scheduled time;

[0037] The calculated information throughput rate E is compared with the set lower limit threshold F, and if E≥F does not meet the requirement, the bandwidth amount is automatically increased;

[0038] Use segmented transmission technology to divide the overly large information flow M into independent units, and then determine whether the retransmission conditions are met based on each sub-packet in turn. i |≤T6, where E i is the energy intensity of the ith small data packet. If the intensity is less than the threshold T6, the automatic retransmission mechanism will be activated.

[0039] Preferably, the vehicle-mounted visual sensor module acquires real-time information of the surrounding environment further comprising:

[0040] Integrate the three-dimensional space model Z formed by multiple cameras, and establish a full range of vision coverage U based on it;

[0041] Obtain the size S and position vector L of the target object acquired by the sensor;

[0042] If the occlusion ratio of a single object reaches the set conditional formula area ratio: H / T4, where H is the size of the occlusion area and T4 refers to the set standard ratio, the navigation mode is activated;

[0043] Dynamically predict vehicle motion boundariesB.

[0044] Preferably, the target detection and recognition module performs target detection and recognition based on the environmental information acquired by the vehicle-mounted visual sensor module and further includes:

[0045] Obtain the mine scene image frame X captured by the camera and convert it into a grayscale image Y;

[0046] Apply a neural network to label the category T in the grayscale image Y;

[0047] If the absolute value |M| of the object motion vector M exceeds a specific threshold T3, an alarm is triggered to indicate that the vehicle should decelerate until it stops.

[0048] Based on the distance relationship |O1O2| < D between all currently detected targets, where O1 and O2 represent the distance measurement values D of the first and second nearest detected dynamic targets respectively, once this formula is satisfied, an avoidance action is executed.

[0049] Preferably, the path planning module uses real-time updated map data for path planning of the unmanned mining truck and further includes:

[0050] Based on the latest received mine environment change information L, perform path replanning to avoid dynamic obstacle objects Q;

[0051] Calculate the time cost S between different path points;

[0052] Set an obstacle avoidance safety range Z, and avoid when the distance P between predicted path points is less than the obstacle avoidance safety range Z;

[0053] Based on the vehicle speed V, estimate the total driving time G required to complete the task, ensuring that it does not exceed the specified time window T2.

[0054] Preferably, the information synchronization module eliminates information synchronization errors based on the communication characteristics of the 5G network and includes:

[0055] Use the vehicle-mounted antenna array to dynamically adjust the transmission angle A;

[0056] Implement a multi-channel signal synchronization algorithm to enhance stability J;

[0057] Monitor the signal transmission status to provide real-time feedback on the signal strength C;

[0058] Adjust the communication node based on the current vehicle position information D.

[0059] From the above technical solutions, it can be seen that the present invention has the following beneficial effects:

[0060] The 5G-based unmanned mining truck visual recognition and path planning system realizes real-time data transmission through the communication module based on the high-bandwidth and low-latency characteristics of the 5G network. The on-board visual sensor module obtains real-time information of the surrounding environment. The target detection and recognition module detects and recognizes targets based on the environmental information obtained by the on-board visual sensor module. The path planning module uses real-time updated map data to plan the path of the unmanned mining truck. The information synchronization module eliminates information synchronization errors based on the communication characteristics of the 5G network to ensure accurate operation and navigation of the unmanned mining truck, ensure more stable communication transmission in the complex environment of the mine, improve the real-time and coordination of the entire system, ensure that the unmanned mining truck can complete tasks safely and accurately in harsh environments, avoid collision risks, improve mining operation efficiency, and enhance the autonomous decision-making ability of unmanned mining trucks in complex environments. The system is based on information In the event of unstable or partially lost signals, it can respond reasonably and in time to avoid misoperation caused by information delays or transmission errors, further improving the stability and operational safety of unmanned mining trucks. Unmanned mining trucks can operate efficiently without relying on human intervention, greatly reducing the risks and labor costs of manual driving in mining areas, and solving the problem that the existing technology usually has complex terrain in mining workplaces, and obstacles will cause obstruction of 5G signals, thereby affecting the data transmission and communication synchronization between the unmanned mining truck and the central control system. When signal transmission is blocked, information delays or transmission errors will occur, resulting in the unmanned mining truck being unable to receive or send instructions in time, seriously affecting the vehicle's path planning and decision-making process, and information synchronization errors, especially at critical decision-making moments, causing the unmanned mining truck to make incorrect responses, thereby reducing the stability of the entire system and operational reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a schematic diagram of the connection of the system modules of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] like Figure 1 As shown, a 5G-based unmanned mining truck visual recognition and path planning system, the system includes:

[0064] Communication module: Utilize the high bandwidth and low latency characteristics of the 5G network to achieve real-time data transmission between unmanned mining trucks and remote control centers and other unmanned equipment. This ensures that in complex mining environments, the system can respond quickly and process data, reducing the impact of latency on mining truck operations. Through the 5G network, the system can process high-definition video transmission, environmental data, command feedback, etc., to ensure efficient data flow;

[0065] On-board visual sensor module: Multiple sensors (such as cameras, lidar, etc.) installed on unmanned mining trucks obtain information about the surrounding environment of the mining area in real time. This module can capture image data of mining roads, obstacles, pedestrians and other vehicles, and provide this data to subsequent processing modules. The collaborative work of multiple sensors can improve the accuracy and integrity of data acquisition and ensure full coverage of environmental information;

[0066] Target detection and recognition module: This module uses computer vision algorithms to detect and classify targets based on environmental information obtained by the on-board visual sensor module. Its functions include identifying static and dynamic targets in the mining area, such as rocks, potholes, pedestrians or other vehicles. The target detection module can not only determine the location of the target, but also analyze its movement trajectory to further provide a basis for path planning;

[0067] Path planning module: This module performs path planning based on real-time updated map data. Combined with the information provided by the target detection and recognition module, the path planning module can dynamically adjust the driving route of the unmanned mining truck to avoid obstacles or pedestrians and select the optimal driving route. The system can select a smooth and safe driving route in a complex mining environment to ensure the stable and efficient operation of the unmanned mining truck.

[0068] Information synchronization module: Through the 5G communication network, this module can quickly eliminate data synchronization errors between different sensors or devices, ensuring that all modules receive the latest information. Especially in complex environments, accurate time synchronization is essential for accurate operation and navigation of unmanned mining trucks. This module uses high-precision clock synchronization technology to minimize data synchronization errors, ensuring that there will be no delays or deviations in the navigation and operation of unmanned mining trucks.

[0069] The working principle of the system is mainly based on the high-bandwidth and low-latency characteristics of the 5G network, which transmits the environmental data collected by the on-board visual sensor in real time, and combines the algorithm processing results of the target detection and recognition module for path planning. The collaborative work of the communication module and the information synchronization module ensures the accuracy and timeliness of the real-time data stream and guarantees the precise navigation of the unmanned mining truck. The entire system relies on the efficient processing of real-time data and the seamless collaboration between modules to achieve safe and automated operation in complex mining environments. Through the communication advantages of the 5G network, the system greatly reduces the delay in transmitting and processing data, ensuring that the unmanned mining truck can respond quickly after receiving real-time data to avoid accidents. Using advanced computer vision algorithms, the system can accurately identify obstacles or dynamic targets in the mining area, significantly improving the path planning ability of the unmanned mining truck. The path planning module combines the real-time updated map data to dynamically adjust the driving route to ensure that the unmanned mining truck always runs along the optimal path, reducing energy consumption and improving work efficiency. The 5G network eliminates the information synchronization error between sensors, ensuring that every step of the unmanned mining truck is based on the latest data, and reducing operational errors caused by data delays or inaccuracies.

[0070] In areas where 5G networks are not fully covered, other wireless communication technologies such as 4G or local area networks can be used as alternatives. Although some delays may be added, basic data transmission requirements can still be met. Different mining areas may choose different sensor configurations, such as using higher-precision lidar instead of cameras, or adding infrared sensors to cope with work requirements at night or in low-light environments. The target detection and recognition module can choose different algorithms according to different application scenarios. For example, in an open environment with fewer obstacles, a faster but less accurate algorithm can be selected; in a complex environment, a more accurate detection algorithm can be used. The path planning algorithm can be optimized or replaced, such as using a simple straight-line planning method in a flat mining area, and a more advanced terrain-adaptive path planning algorithm in complex terrain. In some mining applications, if the requirements for information synchronization are low, a synchronization method with a larger time synchronization error can be selected to reduce costs or simplify system design.

[0071] The target detection and recognition module detects and recognizes targets based on the environmental information obtained by the vehicle-mounted visual sensor module, including real-time positioning through convolutional neural networks and the YOLO algorithm. The specific formula is:

[0072] × ,

[0073] in, represents the feature map, represents the convolution kernel, represents the input image;

[0074] The cost function in path planning is calculated based on the distance to the surrounding targets and the weight of the target importance, specifically:

[0075] ,

[0076] in, represents the cost function of path planning, Represents the displacement cost of the current target, Indicates the number of targets involved, represents the precision weight of the target operation, represents the importance weight of the target, Indicates the distance to surrounding targets. Indicates the position of the surrounding targets on the horizontal axis, Indicates the position of surrounding targets on the ordinate;

[0077] The autonomous driving function of the unmanned mining truck is mainly achieved by combining the on-board visual sensor module and the 5G communication network. The system first extracts the characteristic information of the surrounding environment through a convolutional neural network, and uses the YOLO algorithm for target detection and recognition. After detecting the target, the path planning module performs optimal path planning based on the real-time updated map data and cost function to ensure that the unmanned mining truck can drive safely and efficiently in the mining area. The 5G communication module ensures the real-time transmission of data, allowing the unmanned mining truck to respond quickly to environmental changes and avoid the impact of delays on driving. Based on the high bandwidth and low latency characteristics of the 5G network, the system can quickly process and transmit large amounts of data, ensuring the real-time response and rapid decision-making capabilities of the unmanned mining truck, greatly improving the operating efficiency. The use of advanced convolutional neural networks and YOLO algorithms enables the system to achieve high-precision target detection and recognition in complex mining environments, accurately determine the location and type of obstacles, pedestrians and other vehicles, and ensure safe operation. Through the optimization of the cost function, the system can comprehensively consider the displacement cost of the target, the importance of the surrounding environment and its distance from the target, plan the optimal driving path, avoid obstacles while optimizing the driving route, reduce energy consumption and improve operating efficiency. The information synchronization module uses the 5G network to eliminate information delays, ensure that the modules in the system can operate synchronously, and ensure that the driving path, target recognition and operation of the unmanned mining truck are based on the latest real-time data, avoiding potential safety hazards caused by data delays.

[0078] The path planning module uses real-time updated map data to plan the path of the unmanned mining truck, including: minimizing the total time by going to each target time, and combining dynamic information to calculate the objective function of path planning, specifically:

[0079] ,

[0080] in, represents the objective function of path planning, Indicates the number of targets involved, Indicates to The time of the target, Indicates The time from one goal to the next goal;

[0081] According to the impact of the environmental information of the surrounding targets on the path, the time to the target is adjusted in real time, specifically:

[0082] ,

[0083] in, Indicates that two targets are at a specific time The actual time elapsed, Indicates time, Indicates the time to reach the target under ideal conditions. Indicates the delay time caused by environmental factors.

[0084] The working principle of the path planning module is based on real-time perception and information feedback of the mining environment. The on-board visual sensor module obtains real-time environmental information, and the target detection and recognition module processes this data and outputs information such as possible obstacles and dynamic targets. Subsequently, the path planning module calculates the optimal path to each target point based on the real-time updated map data and the feedback from target detection, and responds to sudden environmental changes in real time by adjusting the time parameters in the objective function. The system solves the objective function to find the path to achieve the goal in the shortest time. In the path selection process, not only the basic displacement time is taken into account, but also the additional delay caused by the environment is dynamically calculated and adjusted in each path planning. This ensures that the unmanned mining truck can adapt to the dynamic changes of the mining area in real time and improve its operating efficiency and safety. Through the objective function of the path planning module, the system can minimize the total time overhead between multiple target points to ensure that the unmanned mining truck completes the task efficiently. The system dynamically adjusts the time parameters in the path planning based on the real-time updated environmental data, so that it can respond to sudden changes in the complex mining environment in real time and reduce the time delays caused by obstacles, traffic or road conditions. Combined with real-time map data and feedback from visual sensors, the path planning module can accurately select the best path and avoid potential obstacles and dangerous areas, thereby improving the safety of mining operations.

[0085] In a possible implementation, the communication module implements real-time data transmission based on the high bandwidth and low latency characteristics of the 5G network, including the following steps:

[0086] Real-time acquisition of signal blocking areas caused by changes in the mine's geographical environment;

[0087] Dynamically adjust the power of 5G base stations according to the size of the blocked area to ensure uninterrupted communications in key areas;

[0088] Formulate a multi-hop communication link backup strategy based on the location and speed parameters of the unmanned mining truck;

[0089] When it is detected that the signal strength at a certain point on the communication link drops and is lower than the set threshold H, the dynamic data switching process is started, where H represents the set signal strength threshold. When the signal strength drops below A, it indicates that the signal is severely blocked. A represents a set signal strength value, which is used to identify the situation where the signal is severely blocked. A is a signal strength critical value lower than the threshold H. When the signal strength drops below A, it means that the signal of the communication link has been severely affected or blocked, which may cause data transmission interruption or extreme instability. The role of A is to help the system further determine whether the communication quality is in a state of severe attenuation when it detects that the signal strength has dropped to a lower level. If the signal strength is lower than A, the system can take additional emergency measures, such as switching to a backup link, adjusting the 5G base station power, or notifying the operator to intervene.

[0090] The system uses the high bandwidth and low latency characteristics of the 5G network to ensure real-time acquisition of signal obstruction information in complex mining environments, adjust base station power, and ensure smooth communication in key areas. The formulation of the multi-hop communication link backup strategy is based on the real-time location information and speed changes of the vehicle, ensuring that even in the case of signal fluctuations, it can automatically switch to the backup link to ensure the stability of data transmission. When the signal strength of the communication link drops to a certain threshold H, the system immediately starts dynamic switching to prevent complete interruption of the signal. By utilizing the characteristics of the 5G network, the system significantly improves the stability and real-time performance of unmanned mining truck communications, especially in mining environments where signals are easily interfered with or blocked. The ability to automatically adjust base station power and enable backup communication links ensures communication stability in key areas, improves work efficiency and reduces the risk of communication interruption.

[0091] The development of a multi-hop communication link backup strategy includes:

[0092] Monitor the 5G signal strength and its changes around each vehicle;

[0093] Calculate the communication range of all communicable nodes around each vehicle;

[0094] Form a backup route based on the neighboring nodes with the best signal within the communication range;

[0095] It is determined whether the vehicle enters the predefined critical shielding area R. If it enters the area R, the link is triggered to automatically switch to the best backup route to ensure continuous data transmission, where the area R represents a fixed location prone to communication problems.

[0096] By monitoring the changes in 5G signal strength around the unmanned mining truck in real time, the system can detect potential communication problems in advance. When the mining truck enters the predefined signal blocking area R, the system automatically switches to the backup route with the best signal strength to ensure that the communication link is not interrupted. In addition, by calculating the communication nodes around the mining truck and forming redundant routes, the probability of communication failures can be effectively reduced. This implementation method greatly improves the communication continuity of the unmanned mining truck in complex environments, especially when entering the critical signal blocking area. Automatically switching to the optimal route can reduce communication interruptions and ensure the safe operation and efficient operation of the unmanned mining truck.

[0097] The selection range and flexibility of backup routes can be further improved by introducing multi-path transmission protocols. For areas with large signal changes, the communication strategy can be automatically adjusted by predicting signal strength changes in advance, making the system more intelligent and reliable.

[0098] The on-board visual sensor module obtains real-time information about the surrounding environment, including images and depth information around the mining truck through cameras and lidars. The system captures images and depth information of the mining environment in real time through cameras and lidars installed on the mining trucks. The camera is used to obtain two-dimensional visual information around the mining truck, and the lidar is used to generate three-dimensional depth images of objects around the mining truck. By combining image and depth data, the system can understand the environmental conditions around the mining truck in real time and make further driving decisions based on these data. The combination of cameras and lidars can effectively improve the environmental perception capabilities of unmanned mining trucks. The addition of depth information can more accurately detect the distance and shape of surrounding objects, providing more reliable data support for the path planning and obstacle avoidance of unmanned mining trucks. In addition, the ability to obtain environmental information in real time ensures the safety and efficient operation of unmanned mining trucks in complex mining environments.

[0099] The communication module realizes real-time data transmission based on the high bandwidth and low latency characteristics of the 5G network. The implementation method of real-time data transmission based on the 5G network also includes:

[0100] Collect large amounts of information generated by multiple on-board sensors and immediately pre-process them to ensure data availability;

[0101] Evaluate the transmission delay T5 to ensure that critical instructions can be delivered to the destination before the scheduled time;

[0102] The calculated information throughput rate E is compared with the set lower limit threshold F, and E≥F. If it does not meet the minimum requirement, the bandwidth amount will be automatically increased;

[0103] Use segmented transmission technology to divide the overly large information flow M into smaller independent units, and then determine whether the retransmission conditions are met based on each sub-packet in turn. i |≤T6, where E i is the energy intensity of the ith small data packet. If the intensity is less than the threshold T6, the automatic retransmission mechanism will be started. T6 represents the energy intensity threshold used to determine whether a single data packet (subpacket) needs to be retransmitted. The setting of T6 is usually adjusted based on the following factors:

[0104] T6 can be dynamically adjusted according to the channel quality of the 5G network. If the network environment is good and the signal strength is stable, T6 can be set to a lower value; but if the signal interference is large and the channel condition is not ideal, T6 will be set to a higher value to ensure the reliability of transmission;

[0105] For some mission-critical data or high-priority command packets (such as emergency control commands in a mining environment), T6 can be set relatively high to ensure that the data transmission quality can be maintained when the signal weakens. For ordinary data, T6 can be set slightly lower to avoid too frequent retransmissions;

[0106] T6 should also be adjusted according to the requirements of the entire system for latency and data transmission integrity. If the system has high latency requirements, T6 can be set lower so that retransmission is triggered when the signal is slightly weakened to prevent transmission delays. However, if the system allows a certain amount of latency, T6 can be set higher to reduce the number of retransmissions and improve system throughput;

[0107] T6 can also be set based on the historical transmission data and experience of the network. The system can monitor the past data packet transmission, analyze which cases require retransmission, and then dynamically set an optimal T6 threshold based on this data.

[0108] The system first collects a large amount of information around the mining truck through multiple sensors, and performs real-time preprocessing to ensure the validity of the data before transmission. By calculating the information throughput rate E and comparing it with the set minimum threshold F, when the throughput rate is insufficient, the system will automatically increase the 5G network bandwidth to ensure the stability of data transmission. In order to avoid transmission delays caused by excessively large data packets, the system adopts segmented transmission technology to divide the information flow into multiple small units, and judge whether reissue is needed based on the transmission status of each unit to ensure data integrity. Through the efficient data transmission and preprocessing mechanism of the 5G network, the unmanned mining truck can complete the transmission and processing of a large amount of data in a short time, ensuring the real-time and safety of the mining truck's operation. The adoption of segmented transmission technology reduces the blockage of data transmission, improves transmission efficiency, and further ensures the integrity of data transmission through the automatic reissue mechanism.

[0109] In a possible implementation, the on-vehicle vision sensor module obtaining real-time information of the surrounding environment further includes:

[0110] Integrating multiple cameras to form a three-dimensional space model Z, and establishing an omnidirectional vision coverage U based on this;

[0111] Obtaining the size S and position vector L of the target object collected by the sensor;

[0112] If the occlusion ratio of a single object reaches the set condition formula area ratio: H / T4, where H is the size of the occlusion area and T4 refers to the set standard ratio, activate the special navigation mode;

[0113] Dynamically predicting the vehicle movement boundary B and the feasible region range.

[0114] Through the collaborative work of multiple cameras, the system generates a three-dimensional space model Z of the environment around the mining truck and forms an omnidirectional vision U, ensuring a non-blind-spot perception of the environment. The size S of the object obtained by the vision sensor and its position vector L can be used to calculate the occlusion situation of the current object. If the occlusion area H of the object exceeds the set ratio threshold T4, the system will immediately activate the navigation mode to avoid obstacles or adjust the route. At the same time, based on the vehicle movement boundary B and the feasible region, the system dynamically adjusts the driving path to ensure driving safety. Through the establishment of the three-dimensional space model and the prediction of the dynamic movement boundary, the system can more accurately identify surrounding obstacles and potential threats, improving the autonomous navigation ability of the unmanned mining truck. When there are too many occluding objects, automatically activating the special navigation mode can avoid accidental collisions and ensure the driving safety of the unmanned mining truck in complex environments.

[0115] The target detection and recognition module performing target detection and recognition based on the environmental information obtained by the on-vehicle vision sensor module further includes:

[0116] Obtaining the mine scene image frame X captured by the camera and converting it into a grayscale image Y;

[0117] Applying a neural network to label the category T in the grayscale image Y;

[0118] If the absolute value |M| of the object movement vector M exceeds a specific threshold T3, trigger an alarm to indicate that the vehicle decelerates until it stops;

[0119] Based on the distance relationship formula |O1O2| < D between all currently detected targets, calculate whether there is a potential conflict between the targets. Here, O1 and O2 respectively represent the distance measurement values D of the first and second nearest detected dynamic targets. Once this formula is satisfied, perform an avoidance action.

[0120] The system captures the scene image of the mining area through a camera and converts it into a grayscale image to simplify subsequent processing. Then, the neural network analyzes the grayscale image, annotates and identifies the target category. When the motion vector M of the target object exceeds the set threshold T3, the system automatically triggers the warning mechanism and instructs the mining truck to slow down. In addition, by monitoring the distance between targets in real time, if the distance between the two nearest dynamic targets is less than the set threshold D, the system will perform evasive actions to prevent potential collisions. The system uses neural networks to automatically identify and classify targets, and can quickly respond to dynamic changes in the mining area and detect potential threats in a timely manner. When abnormal object movement is detected, the system will respond immediately to ensure the safety of unmanned mining trucks. In addition, through accurate calculation of the distance between targets and conflict detection, the system can effectively avoid collisions between mining trucks and other dynamic objects.

[0121] The path planning module uses real-time updated map data to plan the path of the unmanned mining truck and also includes:

[0122] Replan the path based on the latest received mine environment change information L to avoid dynamic obstacles Q;

[0123] Calculate the time cost S between different path points and optimize the route according to the shortest time principle;

[0124] Set the obstacle avoidance safety range Z, and avoid when the distance P between the predicted path points is less than the obstacle avoidance safety range Z;

[0125] The total driving time G required to complete the task is estimated based on the vehicle speed V to ensure that the specified time window T2 is not exceeded.

[0126] The system dynamically adjusts the driving path of the unmanned mining truck by receiving and updating the environmental change information L of the mining area in real time to avoid emergencies such as dynamic obstacles Q. The system calculates the time cost S between different path points and plans the optimal driving route for the mining truck based on the principle of the shortest time. When the distance P between the estimated driving path and the obstacle is less than the set safety range Z, the system automatically starts the selection of the detour path to ensure the safety of driving. At the same time, the system estimates the overall driving time G required to complete the task based on the vehicle's speed V, and ensures that the time does not exceed the specified time window T2 to ensure the efficient completion of the task. Through real-time updated mining area environmental data and dynamic adjustment of path planning, the system can effectively cope with complex mining area environments and avoid driving delays or dangers caused by obstacles or environmental changes. The optimization algorithm for calculating the shortest path and time cost not only improves work efficiency, but also ensures that the unmanned mining truck completes the transportation task within the specified time. Setting the obstacle avoidance safety range Z further improves the safety of the system and avoids potential collision risks.

[0127] The information synchronization module eliminates information synchronization errors based on the communication characteristics of the 5G network, including:

[0128] Dynamically adjust the transmission angle A using the vehicle-mounted antenna array;

[0129] Implement multi-channel signal synchronization algorithm to enhance stability J;

[0130] Monitor the signal transmission status to provide real-time feedback of the signal strength C, and automatically start the backup communication solution when the signal strength C is lower than a preset value T1; and

[0131] The communication nodes are adjusted based on the current vehicle position information D.

[0132] The system dynamically adjusts the transmission angle A through the vehicle-mounted antenna array to ensure that the signal can be transmitted stably in different environments. At the same time, the system adopts a multi-channel signal synchronization algorithm to enhance the stability of signal transmission. When the signal strength C is detected to drop to the preset threshold T1, the system automatically starts the backup communication solution to avoid communication interruption due to signal attenuation. In addition, based on the real-time location information D of the vehicle, the system can dynamically adjust the distribution of communication nodes so that high-quality communication can be maintained even in complex terrain. By dynamically adjusting the antenna transmission angle and implementing a multi-channel signal synchronization algorithm, the system can effectively reduce errors in information transmission and ensure data synchronization. The ability to monitor signal strength in real time and switch to the backup communication solution in time greatly improves the reliability and stability of communication, especially in mining environments where signals are easily attenuated. In addition, the dynamic adjustment of communication nodes based on the vehicle's position ensures the continuous communication capability of the unmanned mining truck and improves the safety and efficiency of the entire system.

[0133] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A 5G-based unmanned mining truck visual recognition and path planning system, characterized in that: The system comprises: Communication module, used to achieve real-time data transmission based on the high bandwidth and low latency characteristics of 5G network; On-board visual sensor module, used to obtain real-time information about the surrounding environment; The target detection and recognition module is used to detect and recognize targets based on the environmental information obtained by the on-board visual sensor module; The path planning module is used to plan the path of the unmanned mining truck using real-time updated map data; Information synchronization module, which is used to eliminate information synchronization errors based on the communication characteristics of the 5G network to ensure accurate operation and navigation of unmanned mining trucks; The target detection and recognition module detects and recognizes the target based on the environmental information obtained by the vehicle-mounted visual sensor module, including: first, real-time positioning, the specific formula is: × ， in, represents the feature map, represents the convolution kernel, represents the input image; The cost function in path planning is calculated based on the distance to the surrounding targets and the weight of the target importance, specifically: ， in, represents the cost function of path planning, Represents the displacement cost of the current target, Indicates the number of targets involved, represents the precision weight of the target operation, represents the importance weight of the target, Indicates the distance to surrounding targets. Indicates the position of the surrounding targets on the horizontal axis, Indicates the position of surrounding targets on the ordinate; The path planning module uses the map data updated in real time to plan the path of the unmanned mining truck, including: minimizing the total time by going to each target time, and calculating the objective function of the path planning in combination with dynamic information, specifically: ， in, represents the objective function of path planning, Indicates the number of targets involved, Indicates to The time of the target, Indicates The time from one goal to the next goal; According to the impact of the environmental information of the surrounding targets on the path, the time to the target is adjusted in real time, specifically: ， in, Indicates that two targets are at a specific time The actual time elapsed, Indicates time, Indicates the time to reach the target under ideal conditions. Indicates the delay time caused by environmental factors; The communication module realizes real-time data transmission based on the high bandwidth and low latency characteristics of the 5G network, including: Real-time acquisition of signal blocking areas caused by changes in the mine's geographical environment; Adjust the 5G base station power according to the size of the obstructed area to ensure unobstructed communications in key areas; Formulate a multi-hop communication link backup strategy based on vehicle position and speed parameters; The dynamic data switching process is started when it is detected that the signal strength at a certain point on the communication link drops by more than the set threshold H, where H represents the set signal strength threshold. When the signal strength drops below A, it indicates that the signal is severely blocked.

2. According to the 5G-based unmanned mining truck visual recognition and path planning system of claim 1, it is characterized by: The formulation of a multi-hop communication link backup strategy includes: Monitor the 5G signal strength and its changes around each vehicle; Calculate the communication range of all communicable nodes around each vehicle; Form a backup route based on the neighboring nodes with the best signal within the communication range; It is determined whether the vehicle enters the predefined critical shielding area R. If it enters the area R, the link is triggered to automatically switch to the best backup route to ensure continuous data transmission, where the area R represents a fixed location prone to communication problems.

3. According to the 5G-based unmanned mining truck visual recognition and path planning system of claim 1, it is characterized by: The on-board visual sensor module obtains real-time information about the surrounding environment, including obtaining images and depth information around the mining truck through cameras and laser radars.

4. According to the 5G-based unmanned mining truck visual recognition and path planning system of claim 1, it is characterized by: The communication module realizes real-time data transmission based on the high bandwidth and low latency characteristics of the 5G network and also includes: Collect large amounts of information generated by multiple on-board sensors I; Evaluate the transmission delay T5 to ensure that critical instructions can be delivered to the destination before the scheduled time; The calculated information throughput rate E is compared with the set lower limit threshold F, and if E≥F does not meet the requirement, the bandwidth amount is automatically increased; Use segmented transmission technology to divide the overly large information flow M into independent units, and then determine whether the retransmission conditions are met based on each sub-packet in turn. i |≤T6, where E i is the energy intensity of the ith small data packet. If the intensity is less than the threshold T6, the automatic retransmission mechanism will be activated.

5. According to the 5G-based unmanned mining truck visual recognition and path planning system of claim 1, it is characterized by: The vehicle-mounted visual sensor module acquires real-time information of the surrounding environment and further includes: Integrate the three-dimensional space model Z formed by multiple cameras, and establish a full range of vision coverage U based on it; Obtain the size S and position vector L of the target object acquired by the sensor; If the occlusion ratio of a single object reaches the set condition, i.e., the area ratio: H / T4, where H is the size of the occluded area and T4 is the set standard ratio, the navigation mode is activated; Dynamically predict the vehicle motion boundary B.

6. According to claim 1, a 5G-based unmanned mining truck visual recognition and path planning system is characterized by: The object detection and recognition module for detecting and recognizing objects based on the environmental information obtained by the vehicle-mounted vision sensor module further includes: Obtain the mine scene image frame X captured by the camera and convert it into a grayscale image Y; Apply a neural network to label the category T in the grayscale image Y; If the absolute value |M| of the object motion vector M exceeds a specific threshold T3, an alarm is triggered to indicate that the vehicle decelerates until it stops; Based on the distance relationship formula |O1O2| < D between all currently detected targets, where O1 and O2 respectively represent the distance measurement values D of the first and second nearest detected dynamic targets. Once this formula is satisfied, an avoidance action is executed.

7. The 5G-based unmanned mining truck visual recognition and path planning system according to claim 1 is characterized in that: The path planning module for path planning of the unmanned mining truck using real-time updated map data further includes: Based on the latest received mine environment change information L, perform path replanning to avoid dynamic obstacle objects Q; Calculate the time cost S between different path points; Set an obstacle avoidance safety range Z and avoid it when the distance P between predicted path points is less than the obstacle avoidance safety range Z; Estimate the total driving time G required to complete the task based on the vehicle speed V to ensure that it does not exceed the specified time window T2.

8. The 5G-based unmanned mining truck visual recognition and path planning system according to claim 1 is characterized by: The information synchronization module eliminates information synchronization errors based on the communication characteristics of the 5G network, including: Dynamically adjust the transmission angle A using the vehicle-mounted antenna array; Implement a multi-channel signal synchronization algorithm to enhance stability J; Monitor the signal transmission status to provide real-time feedback on the signal strength C; Adjust the communication node based on the current vehicle position information D.

Images