A mine car automatic driving vision guidance follow-up docking system and method

Abstract

The present invention relates to the field of autonomous driving technology, and discloses a mine car autonomous driving visual guidance following docking system and method, comprising a training subsystem and an autonomous driving subsystem, wherein the training subsystem comprises an on-board vision module, a drive control module, a control signal generating module, a training module, an excitation module and a safety management module; the signal output end of the on-board vision module is connected to the signal input end of the excitation module, the signal output end of the excitation module is connected to the signal input end of the training module and the control signal generating module, the signal output end of the training module is connected to the signal input end of the safety management module and the control signal generating module, the signal output end of the control signal generating module is connected to the signal input end of the training module and the drive control module; the present invention can solve the problem of insufficient maneuverability of traditional navigation technology and path planning strategies, and is helpful to improve safe passability in the event of sudden obstacles and autonomous driving capabilities in dynamically changing environments.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a mine car autonomous driving vision-guided following docking system and method. Background Art

[0002] Autonomous driving navigation and positioning methods include satellite navigation, inertial navigation, visual (laser) SLAM navigation, and magnetic pin (tape, QR code) navigation. Autonomous vehicles primarily use path planning algorithms for short-range parking or stopping. For example, graph search algorithms represent the driving map as a grid and use heuristic algorithms to calculate the shortest path from the starting point to the end point; sampling algorithms randomly generate multiple sampling points and search for valid paths in complex environments point by point; and optimization algorithms output an optimal path that satisfies environmental and vehicle constraints by minimizing or maximizing a cost function. Path planning algorithms require manual pre-setting of the regional grid map and the coordinates of the autonomous vehicle's parking or stopping target. These algorithms then calculate the vehicle's trajectory and control strategy. These algorithms are not suitable for special situations such as dynamic adjustments to the target position. Summary of the Invention

[0003] The purpose of the present invention is to provide a mine car automatic driving vision guidance following docking system and method to solve the problem of insufficient maneuverability of traditional navigation technology and path planning strategy.

[0004] In order to achieve the above purpose, the technical solutions adopted are as follows:

[0005] In a first aspect, the present invention provides a mine car automatic driving vision-guided following and docking system, characterized in that the system includes a training subsystem and an automatic driving subsystem, wherein the training subsystem includes an on-board vision module, a drive control module, a control signal generation module, a training module, an excitation module and a safety management module; wherein the signal output end of the on-board vision module is connected to the signal input end of the excitation module, the signal output end of the excitation module is connected to the signal input end of the training module and the control signal generation module, the signal output end of the training module is connected to the signal input end of the safety management module and the control signal generation module, and the signal output end of the control signal generation module is connected to the signal input end of the training module and the drive control module;

[0006] The automatic driving subsystem includes an on-board vision module, a drive control module, a control signal generation module and a safety management module; wherein, the signal output end of the on-board vision module is connected to the signal input end of the control signal generation module and the safety management module, the signal output end of the control signal generation module is connected to the signal input end of the drive control module, and the signal output end of the safety management module is connected to the signal input end of the control signal generation module.

[0007] Furthermore, in the training subsystem, the onboard vision module has a built-in RGB camera, a depth camera and a video pre-processing unit, which are used to obtain RGBD data and send the RGBD data to the excitation module;

[0008] The excitation module has a built-in target detection network and an excitation value calculation unit. The target detection network is used to pre-train the weight parameters for identifying the following and stopping targets and obstacles inside and outside the path based on the input RGBD data, realize target detection of the following and stopping targets and obstacles inside and outside the path, and send the detection result data to the excitation value calculation unit; the excitation value calculation unit is used to calculate and generate the distance data of the mine car deviating from the following and stopping target based on the detection result data, and then calculate the excitation value and docking instruction for approaching the following and stopping target, and calculate the relative distance data between the mine car and obstacles inside and outside the path, generate obstacle avoidance instructions according to the strategy, and transmit them to the training module in real time;

[0009] The control signal generation module has a built-in main prediction network, an action instruction generation strategy unit, an index strategy unit and a mode control unit, and is used to generate control instructions and send them to the drive control module.

[0010] Furthermore, the training module has a built-in target network, an experience replay cache unit, a remote monitoring unit, and an automatic reset control unit. The experience replay cache unit is used to store RGBD data, stimulus values, obstacle avoidance instructions, and docking instructions. The target network is used to generate an expected value of the probability of an action instruction based on the data stored in the experience replay cache unit. The remote monitoring unit is used to send and receive manual remote control instructions. The automatic reset control unit is used to generate a reset action instruction based on the driving action instruction, obstacle avoidance instruction, and docking instruction.

[0011] The driving control module is used to receive a driving action instruction and control the mining vehicle based on the driving action instruction, wherein the driving action instruction includes a steering instruction and a speed instruction;

[0012] The security management module has a built-in prediction supervision network, which obtains training data from the built-in experience replay cache unit of the training module and is trained independently of the main prediction network and the target network;

[0013] In the autonomous driving subsystem, the on-board vision module has a built-in RGB camera, a depth camera, and a video pre-processing unit, which is used to obtain RGBD data and send the RGBD data to the control signal generation module and the safety management module;

[0014] The control signal generation module calculates and generates the probability of the mine car running action instruction, and the action instruction generation strategy unit calculates and generates the mine car driving action instruction;

[0015] The driving control module is used to receive a driving action instruction and control the mining vehicle based on the driving action instruction, wherein the driving action instruction includes a steering instruction and a speed instruction;

[0016] The safety management module has a built-in prediction supervision network, which generates obstacle avoidance / docking instructions based on RGBD data and sends them to the control signal generation module. The control signal generation module readjusts the driving direction action instructions based on the obstacle avoidance instructions, or generates a docking operation action instruction based on the docking instruction.

[0017] Furthermore, the control signal generating module alternately executes the observation mode and the training mode during the training phase;

[0018] When executing the observation mode, RGBD data, stimulus values, obstacle avoidance instructions, and docking instructions are obtained from the stimulus module, the RGBD data is input into the main prediction network, the probability of the mine car driving action instruction is calculated, and the probability of the mine car driving action instruction is input into the action instruction generation strategy unit to calculate and generate the mine car driving action instruction, which is transmitted to the drive control module, and the RGBD data and its corresponding action instruction, stimulus value, obstacle avoidance instruction, and docking instruction data are sent to the experience replay cache unit; at the same time, based on the obstacle avoidance instruction and the docking instruction, the stop observation action instruction is triggered to drive the mine car to stop running and stop transmitting experience data to the experience replay cache unit;

[0019] When executing the training mode, the main prediction network obtains RGBD data and action instruction data from the experience replay cache unit, calculates the probability of the mine car driving action instruction using the RGBD data, combines the action instruction data in the experience replay cache unit, and calculates and generates the action instruction index value through the index strategy unit to carry out the main prediction network training;

[0020] The control signal generation module has a built-in mode control unit to control the action instruction generation mode and control the switching of observation mode and training mode. The action instruction generation mode includes the main prediction network generating action instructions, receiving and sending manual remote control instructions, receiving and sending obstacle avoidance instructions and docking instructions.

[0021] Furthermore, the control signal generation module has a built-in action instruction generation strategy unit, which generates action instructions in the following manner:

[0022] Get the state-action value output by the main prediction network , ,in is the speed value, To turn to value;

[0023] The action instruction generation strategy unit outputs the action instruction as , the custom random action instruction is expressed as , the custom random value is , the action instruction randomly adjusts the threshold to , , the speed command is calculated by the following formula a v and steering instructions a r :

[0024] ;

[0025] in, Indicates the bit index value of the maximum value in the output action instruction. Indicates a custom random speed command. Indicates a custom random turn instruction;

[0026] During the training phase, as the number of training rounds increases, the , making Continuously tending to the dependent state-action value, the action instruction randomly adjusts the threshold The reduction strategy is expressed as:

[0027] ;

[0028] in, is the initial value, is the pre-set minimum value, is the reduction value per training round, epoches is the number of training rounds.

[0029] Furthermore, the training module has a built-in target network that forms a deep Q network with the main prediction network. The target network has the same network structure as the main prediction network. The target network receives the weight parameters of the main prediction network and updates itself according to a predetermined period. The target network provides the expected value of the action instruction probability for the main prediction network training.

[0030] Furthermore, the training module has a built-in experience replay cache unit to store the RGBD data, running action instructions, incentive values, obstacle avoidance instructions and docking instructions transmitted by the control signal generation module; during the training stage, the experience replay cache unit randomly selects experience data in batches, and sends the current frame RGBD data and the current frame corresponding running action instruction data in the experience data to the main prediction network in the control signal generation module; sends the current frame corresponding incentive value and the next frame RGBD data in the experience data to the target network; the experience replay cache unit randomly selects experience data in batches, and sends the current frame RGBD data and the current corresponding obstacle avoidance instructions and docking instructions in the experience data to the safety management module.

[0031] The experience replay cache unit sets a maximum storage capacity. When the storage capacity exceeds the maximum storage capacity, the experience replay cache unit deletes the earliest experience data and stores new data, always maintaining the maximum storage capacity. If the obstacle avoidance instruction and / or the docking instruction is a valid value, the experience replay cache unit suspends storing the new data.

[0032] Furthermore, the training module has a built-in automatic reset control unit, which stores the driving action instruction signal sequence sent by the control signal generation module from the starting position, and receives the obstacle avoidance instruction and the docking instruction sent by the excitation module. The obstacle avoidance instruction and the docking instruction trigger the automatic reset control unit to send a reset action instruction to the control signal generation module in the reverse order of the cached action instruction signal sequence, and send it to the drive control module through the control signal generation module. The reset action instruction includes a motion action instruction and a rotation action instruction. The motion action instruction is the negative value of the original instruction, and the rotation action instruction is the original instruction, driving the mine car back to the starting position.

[0033] Furthermore, the excitation module has a built-in target detection network, which calculates the coordinates of the upper left and lower right corners of the detection frame of the following and docking target object and the obstacles inside and outside the path relative to the upper left corner of the output RGB image data, and sends them to the excitation value calculation unit;

[0034] The incentive value calculation unit is used to perform incentive value calculation and obstacle avoidance / docking value calculation to obtain incentive values, obstacle avoidance instructions and docking instructions;

[0035] The incentive value calculation includes:

[0036] The coordinates of the upper left and lower right corners of the RGB image detection box of the target object are expressed as and , then the minimum distance between the RGB image detection target frame and the D depth map is ,but:

[0037] ;

[0038] in, Indicates the depth map D The distance value of the coordinate point;

[0039] The incentive value is calculated using the following formula:

[0040] ;

[0041] in is the incentive value, is the empirical coefficient, arctg is the inverse tangent function;

[0042] The obstacle avoidance / docking value calculation includes:

[0043] In the RGBD image output by the vehicle vision module, the forward path of the mine car is established. x q Indicates the displacement of the left edge of the forward path relative to the left side of the image, x t Indicates the forward path width.

[0044] The coordinates of the upper left and lower right corners of the RGB image detection box of the following target object inspection are expressed as and , the coordinates of the upper left and lower right corners of the obstacle RGB image detection frame are expressed as and ;

[0045] Calculate D depth map Corresponding path coverage decision value :

[0046] ;

[0047] in, Indicates a preset maximum value;

[0048] According to the coordinates of the upper left and lower right corners of the detection frame of the following docked target and , calculate the minimum distance value of the target object:

[0049] ;

[0050] According to the coordinates of the upper left and lower right corners of the obstacle detection frame and , calculate the minimum distance to the obstacle:

[0051] ;

[0052] Assume that the output value of obstacle avoidance is b , the docking output value is ,but:

[0053] ;

[0054] in, is the minimum distance threshold between the minecart and the obstacle, It is the minimum distance threshold from the docking target; when the minimum distance between the mine car and the obstacle is less than the minimum threshold, the obstacle avoidance instruction is output; when the minimum distance between the mine car and the target is less than the minimum threshold, the docking instruction is output.

[0055] Furthermore, in the training phase, the deep Q network consisting of the target detection network in the excitation module, the prediction supervision network in the safety management module, the main prediction network in the control signal generation module, and the target network in the training module is trained respectively;

[0056] The target detection network is pre-trained; during training, the mine car is manually controlled to run, and the on-board vision module shoots forward to follow the docked target object and nearby obstacles inside and outside the path, outputs RGBD data, divides and establishes training data sets and validation data sets, and conducts the target detection network training. After completing the target detection network training and performance verification test, the target detection network outputs the incentive value and obstacle avoidance and docking instructions, which are applied to the prediction supervision network and the deep Q network training;

[0057] The predictive supervision network is a classification network and is trained independently of the deep Q network. During training, the safety management module randomly obtains experience data in batches from the experience replay cache unit. The experience data includes RGBD data and its corresponding obstacle avoidance instructions and docking instructions. The obstacle avoidance instructions and docking instructions are used as RGBD data classification labels, and training and validation data sets are established to carry out the predictive supervision network. After completing the training and performance verification test of the predictive supervision network, the predictive supervision network outputs obstacle avoidance / docking instructions, which are used for obstacle avoidance or docking during the autonomous driving stage.

[0058] The deep Q network training phase is divided into an observation mode and a training mode; the observation mode and the training mode are performed alternately;

[0059] In the observation mode, the control signal generation module obtains RGBD data from the vehicle vision module, the main prediction network calculates and generates the probability of the mine car driving action instruction, inputs the action instruction generation strategy unit to calculate and generate the action instruction, and sends it to the drive control module to drive the mine car automatically. The RGBD data and the corresponding action instruction data are simultaneously sent to the experience replay cache unit. In the observation mode, the main prediction network does not perform network weight parameter training;

[0060] In training mode, the Deep Q Network is trained as follows:

[0061] Initialize the network weight parameters of the main prediction network and the target network respectively;

[0062] The main prediction network calculates and outputs the probability of action instructions: The control signal generation module randomly obtains experience data in batches from the experience playback buffer unit. The experience data includes the RGBD data of the current frame and the action instruction data corresponding to the current frame. The current frame RGBD data is input into the main prediction network to calculate the probability of the mine car driving action instruction;

[0063] The index strategy unit calculates and outputs the action instruction index value;

[0064] The target network calculates and outputs the cumulative incentive;

[0065] Calculate the loss function based on the action instruction probability index value and the cumulative incentive; the loss function includes running speed loss and steering loss;

[0066] Main prediction network weight update: Use speed loss, steering loss and Adam gradient descent optimization algorithm to perform backpropagation on the main prediction network and update the network weight parameters;

[0067] Target network weight update: Set the target network weight parameter update cycle. After the main prediction network training times reaches the number of cycles, the main prediction network weight parameters are input into the target network, and the target network weight parameters are updated to be the same as the main prediction network.

[0068] The second invention provides a method for following and docking a mine car with automatic driving and visual guidance, based on the above-mentioned mine car automatic driving and visual guidance following and docking system, the method includes:

[0069] Obtain RGBD data through the vehicle vision module;

[0070] Establish an incentive module, and build a target detection network and an incentive value calculation unit into the incentive module. The target detection network is used to pre-train the weight parameters for identifying the following and stopping target objects and the adjacent obstacles inside and outside the path based on the input RGBD data, perform target detection on the following and stopping target objects and the adjacent obstacles inside and outside the path, and send the detection result data to the incentive value calculation unit; the incentive value calculation unit is used to calculate and generate the distance data of the mine car deviating from the following and stopping target object based on the detection result data, and then calculate the incentive value and the docking instruction for the approaching stop target object, and calculate the relative distance data between the mine car and the adjacent obstacles inside and outside the path, generate the obstacle avoidance instruction according to the strategy, and transmit it to the training module in real time;

[0071] Establish a control signal generation module, and build a main prediction network, an action instruction generation strategy unit, an index strategy unit and a mode control unit in the control signal generation module to generate control instructions and send them to the drive control module. The mode control unit controls the action instruction generation mode and switches between the observation mode and the training mode.

[0072] Establish a training module, and build a target network, an experience replay cache unit, a remote monitoring unit, and an automatic reset control unit into the training module. The experience replay cache unit is used to store RGBD data, stimulus values, obstacle avoidance instructions, and docking instructions. The target network is used to generate an expected value of the probability of an action instruction based on the data stored in the experience replay cache unit. The remote monitoring unit is used to send and receive manual remote control instructions. The automatic reset control unit is used to generate a reset action instruction based on the driving action instruction, obstacle avoidance instruction, and docking instruction.

[0073] Establishing a drive control module, the drive control module is used to receive control instructions and control the operation of the mining vehicle based on the control instructions;

[0074] A security management module is established, and a prediction supervision network is built into the security management module. The prediction supervision network is trained independently of the main prediction network and the target network. During training, the prediction supervision network obtains training data from the experience replay cache unit.

[0075] Furthermore, the main prediction network generates a rotation action instruction probability, and the action instruction generation strategy unit generates a rotation action instruction. The action instruction generation strategy is as follows:

[0076] Assume that the probability of the main prediction network generating a rotation action instruction is , the action instruction generation strategy outputs the steering instruction as , the custom random steering instruction is expressed as , the custom random value is , the steering command random adjustment threshold is , ,but:

[0077] ;

[0078] After the control signal generation module receives the obstacle avoidance instruction sent by the safety management module, the obstacle avoidance method is as follows:

[0079] Randomly adjust the threshold of the action command Increases, the randomness of the output steering command increases, and the mine car is driven to reselect the driving direction that effectively avoids obstacles.

[0080] Furthermore, the autonomous driving subsystem is constructed by the following method:

[0081] Obtain RGBD data through the vehicle vision module;

[0082] Establish a control signal generation module, and build a main prediction network and action instruction generation strategy unit into the control signal generation module. Generate control instructions based on the RGBD data sent by the vehicle vision module and send them to the drive control module; and receive obstacle avoidance instructions or docking instructions from the safety management module;

[0083] Establishing a drive control module, the drive control module is used to receive control instructions and control the operation of the mining vehicle based on the control instructions;

[0084] A safety management module is established, and a prediction supervision network is built into the safety management module to generate obstacle avoidance instructions or docking instructions based on the RGBD data sent by the on-board vision module and send them to the control signal generation module.

[0085] The beneficial effects of the present invention are:

[0086] Based on theoretical research and experimental verification in a simulated real-world environment, this paper establishes a DQN network training environment and an autonomous driving navigation and obstacle avoidance control system in real-world scenarios in open-pit mines. Targeting unmanned transportation operations in open-pit mines, this paper effectively compensates for the lack of mobility of traditional navigation technologies and path planning strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It shows an operating framework diagram of a training subsystem of a mine car automatic driving vision-guided following docking system according to an embodiment of the present invention;

[0088] Figure 2 It shows a structural diagram of a vehicle-mounted vision module according to an embodiment of the present invention;

[0089] Figure 3 A schematic diagram of an RGBD data flow according to an embodiment of the present invention is shown;

[0090] Figure 4 shows a structural diagram of a control signal generating module according to an embodiment of the present invention;

[0091] Figure 5 shows a structural diagram of a training module according to an embodiment of the present invention;

[0092] Figure 6 A schematic diagram of an experience data transmission path according to an embodiment of the present invention is shown;

[0093] Figure 7 It shows a structural diagram of an excitation module according to an embodiment of the present invention;

[0094] Figure 8 A diagram showing the operating framework of a mine car automatic driving vision-guided following and docking system in an automatic driving subsystem according to an embodiment of the present invention is shown;

[0095] Figure 9 shows a deep Q network model architecture diagram according to an embodiment of the present invention;

[0096] Figure 10 shows a structural diagram of a preprocessing module according to an embodiment of the present invention;

[0097] Figure 11 shows a structural diagram of a residual module according to an embodiment of the present invention;

[0098] Figure 12 shows a residual unit structure diagram according to an embodiment of the present invention;

[0099] Figure 13 A schematic diagram of data flow during the deep Q network training process according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0100] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0101] The specific implementation of the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0102] Example 1:

[0103] The Deep Q-Network (DQN) is an algorithm based on deep learning and reinforcement learning, designed to solve Markov decision processes (MDPs) in discrete action spaces. DQN uses two neural networks: a primary network (Q-network) and a target network (Target Q-network). The primary network selects actions, while the target network calculates target Q values. Currently, Deep Q-Networks have achieved excellent performance in intelligent control of Atari games and intelligent decision-making in chess games. In the field of autonomous driving, research has been conducted on following real-world vehicle trajectories on open streets, automatic lane changes, and autonomous driving in simulated urban road scenarios. Training Deep Q-Networks requires a large amount of empirical data, collected in an environment where potentially unsafe maneuvers can be performed. This poses a significant challenge in providing such data in the real world.

[0104] Based on this, an embodiment of the present invention provides a vision-guided, autonomous mine cart docking system. This system, based on a DQN reinforcement learning network, consists of two subsystems: a training subsystem for the training phase and an autonomous driving subsystem for the autonomous driving phase. The training subsystem trains the network models or modules required in the system. The trained weight parameters are then applied to the corresponding network models in the autonomous driving subsystem, enabling vision-guided, autonomous docking of the mine cart.

[0105] like Figure 1 The figure shows the operational framework of the training subsystem of a vision-guided, follow-and-docking system for autonomous mine vehicles. This subsystem includes an onboard vision module, a drive control module, a control signal generation module, a training module, an excitation module, and a safety management module. The modules are connected as follows: the signal output of the vision module is connected to the signal input of the excitation module, which is then connected to the signal inputs of the training module and the control signal generation module. The signal output of the training module is connected to the signal inputs of the safety management module and the control signal generation module, and the signal output of the control signal generation module is connected to the signal inputs of the training module and the drive control module. The onboard vision module acquires RGBD data and feeds it to the excitation module. Based on the RGBD data, the excitation module generates excitation values and obstacle avoidance / docking instructions. These RGBD data, along with the generated excitation values and obstacle avoidance / docking instructions, are fed to the control signal generation module. These excitation values and obstacle avoidance / docking instructions are then fed to the training module. The training module can also acquire manual remote control instructions, which are then fed to the drive control module via the control signal generation module. The training module also exchanges data with the control signal generation module, including RGBD data, action instructions, and network weights. The training module also generates reset action instructions, which are fed directly to the drive control module. The training module also feeds RGBD data, excitation values, and obstacle avoidance (docking) instructions to the safety management module. The control instructions generated by the control signal generation module, such as motion instructions, are fed to the drive control module, which controls the mining vehicle based on the corresponding instructions received from the control signal generation module. The mode control unit in the control signal generation module controls the action command generation mode, including master prediction network generation, reset commands, or manual remote control commands, as well as switching between observation mode and training mode. Manual remote control commands have the highest execution priority, followed by reset commands, and the master prediction network generation command has the lowest priority.

[0106] like Figure 2The figure shows the structure of the vehicle-mounted vision module. The vehicle-mounted vision module has a built-in depth camera, an RGB camera and a video preprocessing unit. The depth camera and the RGB camera respectively shoot the depth video and RGB video in front of the mine car, and input them into the video preprocessing unit synchronously. The video preprocessing unit samples and extracts the frame images from the depth video stream and the RGB video stream at a predetermined frame interval, and preprocesses the depth video frame images using the nearest neighbor interpolation method and the filtering method to fill in and repair the missing pixel values of the depth map caused by black objects, smooth surfaces, transparent objects, parallax effects, etc.; the depth map and RGB map alignment algorithm are used to unify the coordinate system and merge them into a 4-channel RGBD map, as shown in Figure 3 shown.

[0107] The drive control module receives the driving action instructions generated by the control signal generation module, including steering instructions and speed instructions, and converts the steering instructions and speed instructions into electrical signals and sends them to the steering control system and power control system to control the driving state of the mine car.

[0108] like Figure 4 Figure 2 shows the structure of the control signal generation module. The control signal generation module has a built-in main prediction network, action instruction generation strategy unit, index strategy unit, and mode control unit. During the training phase, the control signal generation module is divided into observation mode and training mode.

[0109] In observation mode, the control signal generation module obtains RGBD data, incentive values, and obstacle avoidance / docking instruction data from the excitation module. The RGBD data is fed into the main prediction network, which calculates the probability of the mine cart's driving action instruction. This data is then fed into the action instruction generation strategy unit, which generates the mine cart's driving action instruction data and transmits it to the drive control module. The RGBD data, along with its corresponding action instructions, incentive values, and obstacle avoidance / docking instruction data, are then sent to the experience replay cache unit. Simultaneously, the obstacle avoidance / docking instruction from the excitation module triggers the cessation of observation and action instruction generation, halting the mine cart's operation and ceasing the transmission of experience data to the experience replay cache unit.

[0110] In training mode, the main prediction network obtains RGBD data and action command data from the experience replay cache unit, uses the RGBD data to calculate the probability of the mine car driving action command, combines the action command data of the experience replay cache unit, and calculates and generates the action command index value through the index strategy unit to carry out main prediction network training.

[0111] During the training phase, the control signal generation module can receive manual remote control instructions, trigger the stop observation and generate action instructions, stop driving the mine car, and stop transmitting experience data to the experience replay cache unit.

[0112] like Figure 5The figure shows the result of the training module. The training module has a built-in target network, experience replay cache unit, remote monitoring unit and automatic reset control unit.

[0113] The target network and the main prediction network form a deep Q-network, sharing the same network structure. The target network receives the weight parameters of the main prediction network and updates itself periodically. The target network provides the expected value of the action command probability for the main prediction network training.

[0114] The experience replay buffer unit stores the RGBD data, operation instructions, stimulus values and obstacle avoidance / docking instruction data transmitted by the control signal generation module. Figure 6 As shown, the experience replay cache unit randomly selects experience data in batches and sends the current frame's RGBD data and the corresponding action instruction data to the main prediction network of the control signal generation module. The stimulus value corresponding to the current frame and the RGBD data of the next frame are sent to the target network. Furthermore, the experience replay cache unit randomly selects experience data in batches and sends the current frame's RGBD data and the corresponding obstacle avoidance / docking instruction (3-bit vector) to the safety management module.

[0115] The Experience Replay Cache has a maximum storage capacity. If the maximum storage capacity is exceeded, the Experience Replay Cache will delete the oldest experience data and store new data, always maintaining the maximum storage capacity. If the obstacle avoidance / docking command is valid, the Experience Replay Cache will stop storing new data.

[0116] The remote monitoring unit obtains manual control instructions for the operation of the mine car and sends them to the control signal generation module and the drive control module in turn, which are used to intervene in the operation control during the training phase to avoid dangerous accidents.

[0117] The automatic reset control unit stores the travel action command signal sequence sent by the mine car control signal generation module from the starting position and receives the obstacle avoidance / docking command sent by the excitation module. The obstacle avoidance / docking command triggers the automatic reset control unit to send a travel action command to the drive control module in the reverse order of the cached action command signal sequence. The movement action command is the negative value of the original command, and the rotation action command is the original command, driving the mine car back to the starting position.

[0118] like Figure 7 As shown in the figure, it is the structure diagram of the excitation module. The excitation module has a built-in target detection network and an excitation value calculation unit. Figure 7As shown in the figure, the excitation module obtains RGBD data from the onboard vision module and transmits it to the target detection network. The target detection network pre-trains the weight parameters for identifying the following and docking targets and obstacles inside and outside the path, realizes target detection for these following and docking targets, and sends the detection results to the excitation value calculation unit. Based on the detection results of the target detection network, the excitation value calculation unit calculates the distance data of the mine car from the docking target, and then calculates the excitation value and docking instruction for approaching the docking target. It also calculates the relative distance data between the mine car and obstacles inside and outside the path, generates obstacle avoidance instructions according to the strategy, and transmits them to the training module in real time.

[0119] The security management module has a built-in prediction supervision network, which is a classification network. The prediction supervision network is trained independently of the main prediction network and the target network. The prediction supervision network obtains training data from the experience replay cache unit.

[0120] like Figure 8 Figure 2 shows the operational framework of the autonomous driving subsystem for a vision-guided, follow-and-docking system for autonomous mine carts. This subsystem comprises an onboard vision module (including an RGB camera, a depth camera, and a video preprocessing unit), a drive control module, a control signal generation module (including a main prediction network, an action command generation strategy unit, and a mode control unit), and a safety management module (a prediction and supervision network). The network weight parameters of the autonomous driving subsystem are provided by the corresponding network modules of the training subsystem. During the autonomous driving phase, the training subsystem does not control the mine cart's motion and continuously optimizes the weight parameters of each network. The onboard vision module's depth and RGB cameras synchronously capture depth and RGB video of the mine cart's forward direction. The video preprocessing unit synthesizes and outputs a four-channel RGBD time series data stream, which is then sent to the control signal generation module and the safety management module. The control signal generation module calculates the probability of the mine cart's motion commands, and the action command generation strategy unit calculates the mine cart's motion commands. The drive control module receives the motion commands generated by the control signal generation module and controls the mine cart's driving state. The safety management module has a built-in prediction and supervision network, which generates obstacle avoidance / docking instructions based on RGBD data and sends the docking instructions to the control signal generation module. The built-in mode control unit in the control signal generation module adjusts the control strategy for the mine car operation and drives the mine car to park; it also sends obstacle avoidance instructions to the control signal generation module to readjust the driving direction.

[0121] The following section describes the specific architecture and mechanism of the neural network model involved in the vision-guided docking system for autonomous mine carts. The neural network model architecture used in this system includes a deep Q-network architecture, an object detection network, and a predictive supervision network.

[0122] The deep Q network includes a main prediction network and a target network. The main prediction network and the target network have the same network structure. The target network receives the weight parameters of the main prediction network and updates itself according to a predetermined period.

[0123] The main prediction network uses the ResNet residual network as the baseline network, redesigns the feature classification head, and adopts a two-way feature classification output, such as Figure 9 The specific design method is as follows: the main prediction network inputs RGBD 4-channel data, which is converted into Data, such as Figure 10 As shown. Input 4 residual network modules (ResidualModel) in sequence and output The data is then passed through the average pooling layer and output The data is then passed through two fully connected layers and a SoftMax layer, outputting two The probability data represent the speed probability and steering probability of the mine car action command, and finally input the action command generation strategy to generate the speed command and steering command. Among them, the residual network module contains n residual units in series, such as Figure 11 and Figure 12 shown.

[0124] The action instruction generation strategy unit generates action instructions based on the output of the main prediction network. Let the state-action value output of the main prediction network be expressed as , , including the speed value and turn value , where the speed value The format is shown in Table 1.

[0125] Table 1 Speed value Format

[0126] No. 0 No. 1 No. 2 Speed+ Speed maintenance speed-

[0127] Turning to value The format is shown in Table 2.

[0128] Table 2 Turning Value Format

[0129] No. 0 No. 1 No. 2 Turn left + Steering hold Turn right+

[0130] Assume that the action instruction generation strategy outputs the action instruction represented as , the custom random action instruction is expressed as , the custom random value is , the action instruction randomly adjusts the threshold to , ,but:

[0131] ;

[0132] in, Indicates the bit index value of the maximum value in the output action instruction. Indicates a custom random speed command. Indicates a custom random turn instruction;

[0133] During the training phase, as the number of training rounds increases, the , making Continuously trending towards action-value dependence Action command randomly adjusts the threshold The reduction strategy is expressed as:

[0134] ;

[0135] in, is the initial value, is the pre-set minimum value, is the reduction value per training round, epoches is the number of training rounds.

[0136] During the autonomous driving phase, if the control signal generation module receives the obstacle avoidance command sent by the safety management module, it increases the steering command random adjustment threshold. , the probability of random adjustment of steering instructions increases, mainly by adjusting the steering and reselecting the effective driving direction to avoid obstacles in the driving path.

[0137] The target detection network can adopt the single-stage target detection YOLO series network, for example, YOLOv5, and add an improved spatial channel hybrid attention mechanism network module after the SPPF network layer in the backbone network.

[0138] The object detection network outputs the RGBD data stream, including the coordinates of the upper left and lower right corners of the target detection box and the obstacle detection box inside and outside the path relative to the upper left corner of the RGB frame image. This output is sent to the stimulus value calculation unit, which performs stimulus value calculations and obstacle avoidance / docking value calculations.

[0139] The specific process of stimulus value calculation is as follows:

[0140] Assume that the coordinates of the upper left and lower right corners of the RGB image detection frame of the target object are expressed as and , then the minimum distance between the RGB image detection target frame and the D depth map is ,but:

[0141] ;

[0142] in, Indicates the depth map D The distance value of the coordinate point;

[0143] The incentive value is:

[0144] ;

[0145] in is the incentive value, is the empirical coefficient, arctg is the inverse tangent function.

[0146] Empirical coefficient Make It can be distributed more evenly in the interval [0,1], rather than being biased towards 0 or 1.

[0147] Based on the characteristics of the inverse tangent function, we can analyze that as the distance between the minecart and the docking target increases, the r value decreases. At the same time, after the distance increases to a certain extent, the rate of decrease of the r value decreases, indicating that even if the minecart is far away from the docking target, the incentive value is not too small. Conversely, as the distance between the minecart and the docking target decreases, the r value increases. At the same time, as the minecart and the docking target get closer, the r value increases rapidly, indicating that the minecart receives a larger positive incentive (reward) when it is close to the docking target.

[0148] The specific process of obstacle avoidance / docking value calculation is as follows:

[0149] When the distance between the minecart and the obstacle is lower than a certain threshold, the obstacle avoidance action is triggered; when the distance between the minecart and the docking target is lower than a certain threshold, the docking stop is triggered.

[0150] In the RGBD image output by the vehicle vision module, the forward path of the mine car is established. x q Indicates the displacement of the left edge of the forward path relative to the left side of the image, x t Indicates the forward path width.

[0151] The coordinates of the upper left and lower right corners of the RGB image detection box of the following target object inspection are expressed as and , the coordinates of the upper left and lower right corners of the obstacle RGB image detection frame are expressed as and .

[0152] Calculate D depth map Corresponding path coverage decision value :

[0153] ;

[0154] in, Indicates a preset maximum value;

[0155] According to the coordinates of the upper left and lower right corners of the detection frame of the following docked target and , calculate the minimum distance value of the target object:

[0156] ;

[0157] According to the coordinates of the upper left and lower right corners of the obstacle detection frame and , calculate the minimum distance to the obstacle:

[0158] ;

[0159] Assume that the output value of obstacle avoidance is b , the docking output value is ,but:

[0160] ;

[0161] in, is the minimum distance threshold between the minecart and the obstacle, It is the minimum distance threshold from the docking target; when the minimum distance between the mine car and the obstacle is less than the minimum threshold, the obstacle avoidance instruction is output; when the minimum distance between the mine car and the target is less than the minimum threshold, the docking instruction is output.

[0162] The prediction supervision network uses a ResNet50 classification network, with an improved spatial channel hybrid attention mechanism network module added between the last residual module and the average pooling layer. The prediction supervision network outputs a three-bit vector representing the normal driving, obstacle avoidance instruction bit, and docking instruction bit.

[0163] With the specific structures of the Deep Q Network architecture, the object detection network, and the predictive detection network clearly understood, the following details the training process for these networks. During the training phase, the object detection network, the predictive supervision network, and the Deep Q Network are trained separately.

[0164] Object detection network training:

[0165] The object detection network is pre-trained. During the training phase, the mine cart is manually controlled, and the vehicle's built-in depth camera and RGB camera simultaneously capture forward-facing images of objects following and stopping, as well as obstacles within and outside the path. Training and validation datasets are established, and the object detection network is trained to achieve accurate detection of objects following and stopping, as well as obstacles within and outside the path.

[0166] After completing the target detection network training and performance verification test, the target detection network outputs the stimulus value and obstacle avoidance / docking instructions, which are used for the prediction supervision network and deep Q network training.

[0167] Predictive supervised network training:

[0168] The prediction supervision network is a classification network and is trained independently of the deep Q network (main prediction network and target network). During the training phase, the safety management module obtains batches of random experience data from the experience replay buffer. The experience data includes RGBD data and its corresponding obstacle avoidance / docking instructions. The obstacle avoidance / docking instructions are used as data classification labels. The RGBD data and its corresponding data classification labels are constructed into a data dictionary, and the batch data dictionary is constructed into a data list. The training process includes the following steps:

[0169] Step 10: Equalization processing of sample experience data.

[0170] Set the classification label format as shown in Table 3.

[0171] Table 3 Classification label format

[0172]

[0173] Step 10 can be used to perform sample experience data balancing through the following steps:

[0174] Step 11: Retrieve the data dictionary with the data classification label [0, 0, 1] from the empirical data dictionary list and calculate the number of data dictionaries;

[0175] Step 12: Randomly extract the same number of data dictionaries with data classification labels of [1, 0, 0] and [0, 1, 0] from the empirical data dictionary list;

[0176] Step 13: Perform data augmentation on the RGBD data in the data dictionary extracted in steps 11 and 12, keep the data classification label value unchanged, and expand the number of elements in the data dictionary list to a predetermined number;

[0177] Step 14: sort and shuffle the data dictionary list after data amplification.

[0178] Step 20: Divide the data dictionary list into a training set and a validation set according to a predetermined ratio.

[0179] Step 30: Conduct prediction supervision network training.

[0180] After completing the prediction supervision network training and performance verification test, the prediction supervision network outputs obstacle avoidance / docking instructions, which are applied to the automatic driving stage of the mine car.

[0181] Deep Q network training. Figure 13 As shown in Figure 2, the deep Q network training phase is divided into observation mode and training mode. The observation mode and training mode are performed alternately.

[0182] In the observation mode, the control signal generation module obtains RGBD data from the vehicle vision module, the main prediction network calculates and generates the probability of the mine car driving action command, inputs the action command generation strategy unit to calculate and generate the action command, and sends it to the drive control module to drive the mine car automatically. At the same time, the RGBD data and the corresponding action command data are sent to the experience playback cache unit, such as Figure 13 shown.

[0183] In observation mode, the main prediction network does not perform network weight parameter training.

[0184] In training mode, the Deep Q Network is trained by the following steps:

[0185] Step 1: The main prediction network and the target network are first initialized with network weight parameters.

[0186] Step 2: The main prediction network calculates the output action instruction probability.

[0187] The control signal generation module obtains batch random experience data from the experience playback cache unit, such as Figure 13 As shown in Figure 2, the empirical data includes the current frame RGBD data and the action command data corresponding to the current frame. The current frame RGBD data is input into the main prediction network to calculate the probability of the mine car driving action command.

[0188] Step 3: The index strategy unit calculates and outputs the action instruction index value.

[0189] The action command probability and the action command data corresponding to the current frame are input into the index strategy unit, which calculates and outputs the driving action command speed probability index value and the steering probability index value respectively. The index algorithm is as follows:

[0190] Assume that the action instruction data corresponding to the current frame is expressed as , the main prediction network generates the action instruction speed probability expressed as , the turning probability is expressed as , concatenated row by row , The output action command speed probability index value is represented by , the turning probability index value is expressed as ,but:

[0191] ;

[0192] Step 4: The target network calculates and outputs the cumulative incentive.

[0193] The target network obtains batches of random experience data from the experience replay cache unit, such as Figure 13 As shown, the empirical data includes the stimulus value corresponding to the current frame and the RGBD data of the next frame. The RGBD data of the next frame is input into the target network to calculate the probability of the mine car driving action command, including the speed probability and the steering probability of the action command. The action command probability and the stimulus value corresponding to the current frame are input into the cumulative incentive strategy unit to calculate the target cumulative incentive value of the speed and steering target of the mine car driving action command. The cumulative incentive is as follows:

[0194] Assume that the current frame corresponding to the stimulus value is expressed as , ,in, represents speed incentive, Represents the steering incentive, and the mine car driving action instruction speed probability is expressed as , the turning probability is expressed as , then the speed target cumulative incentive value is expressed as:

[0195] ;

[0196] The cumulative incentive value of the steering target is expressed as:

[0197] ;

[0198] in, is the empirical coefficient.

[0199] Step 5: Calculate the loss function based on the action instruction probability index value and the cumulative incentive.

[0200] The speed loss function is:

[0201] ;

[0202] The steering loss function is:

[0203] ;

[0204] in, is the number of training samples in each round.

[0205] Step 6: Update the weights of the main prediction network.

[0206] Use speed loss, steering loss and Adam gradient descent optimization algorithm to perform backpropagation on the main prediction network and update the network weight parameters.

[0207] Step 7: Update the target network weights.

[0208] Set the target network weight parameter update cycle. After the main prediction network is trained for a certain number of cycles, the main prediction network weight parameters are input into the target network, and the target network weight parameters are updated to be the same as the main prediction network.

[0209] The above steps 2 to 7 are executed cyclically to continuously reduce the loss of the predicted motion speed command and the steering command.

[0210] Example 2:

[0211] An embodiment of the present invention further provides a method for automatic driving of a mine car with visual guidance and following docking, the method comprising:

[0212] Step S1, obtaining RGBD data stream through the vehicle-mounted vision module;

[0213] Step S2: Establish an excitation module, and build a target detection network and an excitation value calculation unit into the excitation module. The target detection network is used to pre-train the weight parameters for identifying the following and stopping targets and obstacles inside and outside the path based on the input RGBD data stream, realize target detection of the following and stopping targets and obstacles inside and outside the path, and send the detection result data to the excitation value calculation unit; the excitation value calculation unit is used to calculate and generate the distance data of the mine car deviating from the following and stopping target based on the detection result data, and then calculate the excitation value and docking instruction for approaching the stopping target, and calculate the relative distance data between the mine car and obstacles inside and outside the path, generate obstacle avoidance instructions according to the strategy, and transmit them to the training module in real time;

[0214] Step S3, establishing a control signal generation module, and incorporating a main prediction network, an action instruction generation strategy unit, an index strategy unit, and a mode control unit into the control signal generation module to generate a control instruction and send it to the drive control module;

[0215] Step S4: Establish a training module, and embed a target network, an experience replay cache unit, a remote monitoring unit, and an automatic reset control unit in the training module. The experience replay cache unit is used to store RGBD data streams, stimulus values, obstacle avoidance instructions, and docking instructions. The target network is used to generate an expected value of the probability of an action instruction based on the data stored in the experience replay cache unit. The remote monitoring unit is used to send and receive manual remote control instructions. The automatic reset control unit is used to generate a reset action instruction based on the driving action instruction, obstacle avoidance instruction, and docking instruction.

[0216] Step S5: establishing a drive control module, wherein the drive control module is used to receive control instructions and control the operation of the mining vehicle based on the control instructions;

[0217] Step S6: Establish a safety management module and build a prediction supervision network into the safety management module. The prediction supervision network is trained independently of the main prediction network and the target network. During training, the prediction supervision network obtains training data from the experience replay cache unit. In the automatic driving stage, the prediction supervision network generates obstacle avoidance / docking instructions based on RGBD data, sends the docking instructions to the control signal generation module, adjusts the motion signal generation strategy, and drives the mine car to park; sends obstacle avoidance instructions to the control signal generation module to readjust the driving direction.

[0218] It should be noted that the mine car automatic driving visual guidance following docking method belongs to the same technical concept as the system described previously, and has the same technical principles and beneficial effects, so it will not be repeated here.

[0219] The above embodiments are only used to illustrate the present invention, and are not intended to limit the present invention. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the present invention. The scope of patent protection of the present invention should be defined by the claims.

Claims

1. A mine car automatic driving vision guidance follow docking system, characterized by: The system includes a training subsystem and an automatic driving subsystem, wherein the training subsystem includes an on-board vision module, a drive control module, a control signal generation module, a training module, an excitation module and a safety management module; wherein the signal output end of the on-board vision module is connected to the signal input end of the excitation module, the signal output end of the excitation module is connected to the signal input end of the training module and the control signal generation module, the signal output end of the training module is connected to the signal input end of the safety management module and the control signal generation module, and the signal output end of the control signal generation module is connected to the signal input end of the training module and the drive control module; The autonomous driving subsystem includes a first vehicle-mounted vision module, a first drive control module, a first control signal generation module, and a first safety management module; wherein the signal output end of the first vehicle-mounted vision module is connected to the signal input ends of the first control signal generation module and the first safety management module, the signal output end of the first control signal generation module is connected to the signal input end of the first drive control module, and the signal output end of the first safety management module is connected to the signal input end of the first control signal generation module; The training module has a built-in automatic reset control unit, which stores the driving action instruction signal sequence sent by the control signal generation module from the starting position, and receives the obstacle avoidance instruction and the docking instruction sent by the excitation module. The obstacle avoidance instruction and the docking instruction trigger the automatic reset control unit to send a reset action instruction to the control signal generation module in the reverse order of the cached action instruction signal sequence, and send it to the drive control module through the control signal generation module. The reset action instruction includes a motion action instruction and a rotation action instruction. The motion action instruction is the negative value of the original instruction, and the rotation action instruction is the original instruction, which drives the mine car back to the starting position.

2. The mine car automatic driving visual guidance following docking system according to claim 1, characterized in that: In the training subsystem, the on-board vision module has a built-in RGB camera, a depth camera and a video pre-processing unit, which are used to obtain RGBD data and send the RGBD data to the excitation module; The excitation module has a built-in target detection network and an excitation value calculation unit. The target detection network is used to pre-train the weight parameters for identifying the following and stopping targets and obstacles inside and outside the path based on the input RGBD data, realize target detection of the following and stopping targets and obstacles inside and outside the path, and send the detection result data to the excitation value calculation unit; the excitation value calculation unit is used to calculate and generate the distance data of the mine car deviating from the following and stopping target based on the detection result data, and then calculate the excitation value and docking instruction for approaching the following and stopping target, and calculate the relative distance data between the mine car and obstacles inside and outside the path, generate obstacle avoidance instructions according to the strategy, and transmit them to the training module in real time; The control signal generation module has a built-in main prediction network, an action instruction generation strategy unit, an index strategy unit and a mode control unit, and is used to generate control instructions and send them to the drive control module.

3. The mine car automatic driving visual guidance following docking system according to claim 2, characterized in that: The training module has a built-in target network, an experience replay cache unit, a remote monitoring unit, and an automatic reset control unit. The experience replay cache unit is used to store RGBD data, stimulus values, obstacle avoidance instructions, and docking instructions. The target network is used to generate an expected value of the probability of an action instruction based on the data stored in the experience replay cache unit. The remote monitoring unit is used to send and receive manual remote control instructions. The automatic reset control unit is used to generate a reset action instruction based on the driving action instruction, obstacle avoidance instruction, and docking instruction. The driving control module is used to receive a driving action instruction and control the mining vehicle based on the driving action instruction, wherein the driving action instruction includes a steering instruction and a speed instruction; The security management module has a built-in prediction supervision network, which obtains training data from the built-in experience replay cache unit of the training module and is trained independently of the main prediction network and the target network; In the autonomous driving subsystem, the first onboard vision module has a built-in RGB camera, a depth camera, and a video preprocessing unit, for acquiring RGBD data and sending the RGBD data to the control signal generation module and the safety management module; The first control signal generating module calculates and generates the probability of the mine car running action instruction, and the action instruction generating strategy unit calculates and generates the mine car driving action instruction; The first drive control module is used to receive a driving action instruction and control the mining vehicle based on the driving action instruction, wherein the driving action instruction includes a steering instruction and a speed instruction; The first safety management module has a built-in predictive supervision network, which generates obstacle avoidance / docking instructions based on RGBD data and sends them to the first control signal generation module. The first control signal generation module readjusts the driving direction action instructions based on the obstacle avoidance instructions, or generates a docking operation action instruction based on the docking instruction.

4. The mine car automatic driving visual guidance following docking system according to claim 3, characterized in that: The control signal generating module alternately executes the observation mode and the training mode during the training phase; When executing the observation mode, RGBD data, stimulus values, obstacle avoidance instructions, and docking instructions are obtained from the stimulus module, the RGBD data is input into the main prediction network, the probability of the mine car driving action instruction is calculated, and the probability of the mine car driving action instruction is input into the action instruction generation strategy unit to calculate and generate the mine car driving action instruction, which is transmitted to the drive control module, and the RGBD data and its corresponding action instruction, stimulus value, obstacle avoidance instruction, and docking instruction data are sent to the experience replay cache unit; at the same time, based on the obstacle avoidance instruction and the docking instruction, the stop observation action instruction is triggered to drive the mine car to stop running and stop transmitting experience data to the experience replay cache unit; When executing the training mode, the main prediction network obtains RGBD data and action instruction data from the experience replay cache unit, calculates the probability of the mine car driving action instruction using the RGBD data, combines the action instruction data in the experience replay cache unit, and calculates and generates the action instruction index value through the index strategy unit to carry out the main prediction network training; The control signal generation module has a built-in mode control unit to control the action instruction generation mode and control the switching of observation mode and training mode. The action instruction generation mode includes the main prediction network generating action instructions, obtaining and sending manual remote control instructions, obtaining and sending obstacle avoidance instructions and docking instructions.

5. The mine car automatic driving visual guidance following docking system according to claim 3, characterized in that: The control signal generation module has a built-in action instruction generation strategy unit, which generates action instructions in the following way: Get the state-action value output by the main prediction network , ,in is the speed value, To turn to value; The action instruction generation strategy unit outputs the action instruction as , the custom random action instruction is expressed as , the custom random value is , the action instruction randomly adjusts the threshold to , , the speed command is calculated by the following formula a v and steering instructions a r : ； in, Indicates the bit index value of the maximum value in the output action instruction. Indicates a custom random speed command. Indicates a custom random turn instruction; During the training phase, as the number of training rounds increases, the , making Continuously tending to the dependent state-action value, the action instruction randomly adjusts the threshold The reduction strategy is expressed as: ； in, is the initial value, is the pre-set minimum value, is the reduction value per training round, epoches is the number of training rounds.

6. The mine car automatic driving visual guidance following docking system according to claim 2, characterized in that: The training module has a built-in target network that forms a deep Q network with the main prediction network. The target network and the main prediction network have the same network structure. The target network receives the weight parameters of the main prediction network and updates itself according to a predetermined period. The target network provides the expected value of the action instruction probability for the main prediction network training.

7. The mine car automatic driving visual guidance following docking system according to claim 1, characterized in that: The training module has a built-in experience replay cache unit that stores the RGBD data, running action instructions, incentive values, obstacle avoidance instructions, and docking instructions transmitted by the control signal generation module. During the training phase, the experience replay cache unit randomly selects experience data in batches, and sends the current frame RGBD data and the running action instruction data corresponding to the current frame in the experience data to the main prediction network in the control signal generation module; sends the current frame corresponding incentive value and the next frame RGBD data in the experience data to the target network; the experience replay cache unit randomly selects experience data in batches, and sends the current frame RGBD data, the current corresponding obstacle avoidance instruction, and the docking instruction in the experience data to the safety management module. The experience replay cache unit sets a maximum storage capacity. When the storage capacity exceeds the maximum storage capacity, the experience replay cache unit deletes the earliest experience data and stores new data, always maintaining the maximum storage capacity. If the obstacle avoidance instruction and / or the docking instruction is a valid value, the experience replay cache unit suspends storing the new data.

8. The mine car automatic driving vision-guided following docking system according to claim 1, characterized in that: The excitation module has a built-in target detection network, which calculates the coordinates of the upper left and lower right corners of the detection frame of the following and docking target object and the obstacles inside and outside the path relative to the upper left corner of the output RGB image data, and sends it to the excitation value calculation unit; The incentive value calculation unit is used to perform incentive value calculation and obstacle avoidance / docking value calculation to obtain incentive values, obstacle avoidance instructions and docking instructions; The incentive value calculation includes: The coordinates of the upper left and lower right corners of the RGB image detection box of the target object are expressed as and , then the minimum distance between the RGB image detection target frame and the D depth map is ,but: ； in, Indicates the depth map D The distance value of the coordinate point; The incentive value is calculated using the following formula: ； in is the incentive value, is the empirical coefficient, arctg is the inverse tangent function; The obstacle avoidance / docking value calculation includes: In the RGBD image output by the vehicle vision module, the forward path of the mine car is established. x q Indicates the displacement of the left edge of the forward path relative to the left side of the image, x t represents the forward path width; The coordinates of the upper left and lower right corners of the RGB image detection box of the following target object are expressed as and , the coordinates of the upper left and lower right corners of the obstacle RGB image detection frame are expressed as and ; Calculate D depth map Corresponding path coverage decision value : ； in, Indicates a preset maximum value; According to the coordinates of the upper left and lower right corners of the detection frame of the following docked target and , calculate the minimum distance value of the target object: ； According to the coordinates of the upper left and lower right corners of the obstacle detection frame and , calculate the minimum distance to the obstacle: ； Assume that the output value of obstacle avoidance is b , the docking output value is ,but: ； in, is the minimum distance threshold between the minecart and the obstacle, It is the minimum distance threshold from the docking target; when the minimum distance between the mine car and the obstacle is less than the minimum threshold, the obstacle avoidance instruction is output; when the minimum distance between the mine car and the target is less than the minimum threshold, the docking instruction is output.

9. The mine car automatic driving visual guidance following docking system according to claim 3, characterized in that: In the training phase, the deep Q network consisting of the target detection network in the excitation module, the prediction supervision network in the safety management module, the main prediction network in the control signal generation module, and the target network in the training module is trained respectively; The target detection network is pre-trained; during training, the mine car is manually controlled to run, and the on-board vision module shoots forward, follows, and stops targets, obstacles inside and outside the path, outputs RGBD data, divides and establishes a training data set and a validation data set, and carries out the target detection network training. After completing the target detection network training and performance verification test, the target detection network outputs an incentive value and obstacle avoidance instructions and a docking instruction, which are applied to the prediction supervision network and the deep Q network training; The predictive supervision network is a classification network and is trained independently of the deep Q network. During training, the safety management module randomly obtains experience data in batches from the experience replay cache unit. The experience data includes RGBD data and its corresponding obstacle avoidance instructions and docking instructions. The obstacle avoidance instructions and docking instructions are used as RGBD data classification labels. A training data set and a validation data set are established to carry out the predictive supervision network. After completing the training and performance verification test of the predictive supervision network, the predictive supervision network outputs obstacle avoidance / docking instructions, which are applied to obstacle avoidance or docking during the autonomous driving stage. The deep Q network training phase is divided into an observation mode and a training mode; the observation mode and the training mode are performed alternately; In the observation mode, the control signal generation module obtains RGBD data from the vehicle vision module, the main prediction network calculates and generates the probability of the mine car driving action instruction, inputs the action instruction generation strategy unit to calculate and generate the action instruction, and sends it to the drive control module to drive the mine car automatically. The RGBD data and the corresponding action instruction data are simultaneously sent to the experience replay cache unit. In the observation mode, the main prediction network does not perform network weight parameter training; In training mode, the Deep Q Network is trained as follows: Initialize the network weight parameters of the main prediction network and the target network respectively; The main prediction network calculates and outputs the probability of action instructions: The control signal generation module randomly obtains experience data in batches from the experience playback buffer unit. The experience data includes the RGBD data of the current frame and the action instruction data corresponding to the current frame. The current frame RGBD data is input into the main prediction network to calculate the probability of the mine car driving action instruction; The index strategy unit calculates and outputs the action instruction index value; The target network calculates and outputs the cumulative incentive; Calculate the loss function based on the action instruction probability index value and the cumulative incentive; the loss function includes running speed loss and steering loss; Main prediction network weight update: Use speed loss, steering loss and Adam gradient descent optimization algorithm to perform backpropagation on the main prediction network and update the network weight parameters; Target network weight update: Set the target network weight parameter update cycle. After the main prediction network training times reaches the number of cycles, the main prediction network weight parameters are input into the target network, and the target network weight parameters are updated to be the same as the main prediction network.

10. A mine car automatic driving vision-guided following docking method, characterized in that: Based on the mine car automatic driving vision-guided following docking system according to any one of claims 1 to 9, the method comprises: Obtain RGBD data through the vehicle vision module; Establish an incentive module, and build a target detection network and an incentive value calculation unit into the incentive module. The target detection network is used to pre-train the weight parameters for identifying the following and stopping targets and the adjacent obstacles inside and outside the path based on the input RGBD data, perform target detection on the following and stopping targets and the adjacent obstacles inside and outside the path, and send the detection result data to the incentive value calculation unit; the incentive value calculation unit is used to calculate and generate the distance data of the mine car deviating from the following and stopping target based on the detection result data, and then calculate the incentive value and the docking instruction for the approaching stop target, and calculate the relative distance data between the mine car and the obstacles inside and outside the path, generate the obstacle avoidance instruction according to the strategy, and transmit it to the training module in real time; Establish a control signal generation module, and build a main prediction network, an action instruction generation strategy unit, an index strategy unit and a mode control unit in the control signal generation module to generate control instructions and send them to the drive control module. The mode control unit controls the action instruction generation mode and switches between the observation mode and the training mode. Establish a training module, and build a target network, an experience replay cache unit, a remote monitoring unit, and an automatic reset control unit into the training module. The experience replay cache unit is used to store RGBD data, stimulus values, obstacle avoidance instructions, and docking instructions. The target network is used to generate an expected value of the probability of an action instruction based on the data stored in the experience replay cache unit. The remote monitoring unit is used to send and receive manual remote control instructions. The automatic reset control unit is used to generate a reset action instruction based on the driving action instruction, obstacle avoidance instruction, and docking instruction. Establishing a drive control module, the drive control module is used to receive control instructions and control the operation of the mining vehicle based on the control instructions; A security management module is established, and a prediction supervision network is built into the security management module. The prediction supervision network is trained independently of the main prediction network and the target network. During training, the prediction supervision network obtains training data from the experience replay cache unit.

11. The mine car automatic driving visual guidance following docking method according to claim 10, characterized in that: The main prediction network generates the probability of the rotation action instruction, and the action instruction generation strategy unit generates the rotation action instruction. The action instruction generation strategy is as follows: Assume that the probability of the main prediction network generating a rotation action instruction is , the action instruction generation strategy outputs the steering instruction as Custom random steering instructions are expressed as , the custom random value is , the steering command random adjustment threshold is , ,but: ； After the control signal generation module receives the obstacle avoidance instruction sent by the safety management module, the obstacle avoidance method is as follows: Randomly adjust the threshold of the action instruction Increases, the randomness of the output steering command increases, and the mine car is driven to reselect the driving direction that effectively avoids obstacles.

12. The mine car automatic driving visual guidance following docking method according to claim 10, characterized in that: The autonomous driving subsystem is constructed as follows: Acquire RGBD data through a first vehicle-mounted vision module; Establish a first control signal generation module, and build a main prediction network and an action instruction generation strategy unit into the first control signal generation module to generate a control instruction based on the RGBD data sent by the first vehicle-mounted vision module and send it to the drive control module; And receive obstacle avoidance instructions or docking instructions from the safety management module; Establishing a first drive control module, wherein the first drive control module is used to receive control instructions and control the operation of the mining vehicle based on the control instructions; A first safety management module is established, and a prediction supervision network is built into the first safety management module. According to the RGBD data sent by the first vehicle-mounted vision module, an obstacle avoidance instruction or a docking instruction is generated and sent to the first control signal generation module.

Images