Loader shoveling dynamic decision-making method based on multi-source data fusion and reinforcement learning

Abstract

The invention relates to a loader shoveling dynamic decision-making method based on multi-source data fusion and reinforcement learning, and belongs to the technical field of engineering machinery automation. According to the method, a material pile feature model is constructed through multi-source data such as a laser radar, a visual sensor and a pressure sensor, the material shoveling path and force are dynamically optimized through a reinforcement learning algorithm, and accurate control over the target material shoveling amount is achieved. According to the method, uncertainty of material pile characteristics is described by using an intuitionistic fuzzy set, a multi-dimensional evaluation result is fused in combination with an evidence theory, and an optimal decision strategy is autonomously learned by virtue of a deep Q network, so that the problem of insufficient shoveling quantity control precision of a traditional method under a dynamic working condition is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of engineering machinery automation, and relates to a loader shoveling dynamic decision-making method based on multi-source data fusion and reinforcement learning. BACKGROUND

[0002] The loader is an indispensable key equipment in mine, port and construction site scenes, and its main task is to shovel and load the bulk material (such as ore, sand, etc.) to the designated position. The shoveling operation of the traditional loader highly depends on the experience and skills of the operator, and there are problems such as unstable operation efficiency, high labor intensity, high energy consumption, etc.

[0003] Under the trend of industrial automation and intelligence, the demand for unmanned shoveling operation of the loader is increasingly urgent. In the process of shoveling the loader, it is of key significance to improve the operation efficiency and reduce the cost to achieve accurate control of the target shoveling amount. However, the bulk material in the actual operation environment has various shapes such as common conical and slope accumulation, and the material characteristics also have significant differences due to different densities, humidities and particle sizes, combined with dynamic factors such as slope changes and obstacle distribution of the operation site, so that the traditional shoveling decision-making method is difficult to achieve accurate control of the target shoveling amount.

[0004] The existing technology mainly has the following limitations:

[0005] (1) Limitations of data acquisition and analysis: Most of the traditional methods only rely on a single sensor, such as only relying on a pressure sensor to monitor the load of the bucket, which cannot comprehensively analyze the multi-dimensional data such as the three-dimensional shape of the material pile and the distribution density of the material. This leads to one-sided decision-making, such as being difficult to predict the loose area inside the material pile in advance through load data, which easily causes insufficient or overloaded shoveling.

[0006] (2) Lack of dynamic adaptability of decision-making model: Fixed rules or simple weighted algorithms are mostly used. Such algorithms are difficult to adjust flexibly when facing complex and variable working conditions, and when dealing with multi-objective decision-making, an ideal solution needs to be preset in advance, but the fixed ideal solution cannot meet the actual demand due to the real-time changes of the material pile shape during the operation of the loader.

[0007] (3) Insufficient processing capacity for uncertain factors: There are errors in sensor measurement itself, such as distance measurement deviation of laser radar in complex environment, and material sliding during shoveling, etc. These uncertain factors will cause fluctuations in the evaluation results. The existing technology lacks systematic modeling of these uncertainties.

[0008] Therefore, how to enable the loader to comprehensively perceive the operation environment like an experienced operator, and dynamically and intelligently adjust the shoveling strategy according to the real-time working conditions to achieve accurate control of the target shoveling amount is a technical problem to be solved at present. SUMMARY

[0009] Therefore, the present application aims to provide a loader shovel dynamic decision-making method based on multi-source data fusion and reinforcement learning to solve the problem of insufficient control accuracy of the traditional method under dynamic working conditions.

[0010] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0011] Scheme 1: A loader shovel dynamic decision-making method based on multi-source data fusion and reinforcement learning, a pile feature model is constructed by multi-source data such as laser radar, visual sensor, pressure sensor, etc., and the reinforcement learning algorithm is used to dynamically optimize the shovel path and intensity, so as to realize the accurate control of the target shovel amount. This method uses intuitionistic fuzzy set to describe the uncertainty of pile characteristics, combines evidence theory to fuse multi-dimensional evaluation results, and uses deep Q network to learn the optimal decision strategy,

[0012] Effectively overcome the problem of insufficient control accuracy of the traditional method under dynamic working conditions.

[0013] The method specifically includes the following steps:

[0014] Step 1: Multi-source state perception: real-time acquisition of state data of the pile and the loader by at least one sensor mounted on the loader; the sensor includes one or more combinations of laser radar, visual sensor, pressure sensor, inclination sensor and position sensor;

[0015] Step 2: Data processing and feature modeling: pre-processing of the collected state data and constructing a feature model describing the working environment; the pre-processing includes filtering and normalization; constructing the feature model includes mapping the processed data to fuzzy language variables using fuzzy logic to construct a fuzzy decision matrix;

[0016] Step 3: Intelligent decision and optimization: input the fuzzy decision matrix and other state parameters as state input into the pre-trained deep reinforcement learning model, and the model outputs the optimal shovel action instruction; the shovel action instruction includes at least one of the forward speed, lifting height and digging angle of the shovel;

[0017] Step 4: Action execution and feedback: control the loader to execute the shovel action instruction, and calculate the reward value according to the difference between the actual digging amount and the target digging amount after execution, and other factors such as working time, and use the reward value to update the deep reinforcement learning model online or offline training.

[0018] Furthermore, in step one, the status data includes: the height, slope, outline coordinates, surface texture, color, and shape of the material pile, as well as the bucket force, material density, loader tilt angle, and loader position coordinates.

[0019] Furthermore, in step two, the filtering process includes: using a median filtering algorithm to denoise the contour coordinate data acquired by the lidar, and / or using a Kalman filtering algorithm to denoise the bucket force data acquired by the pressure sensor.

[0020] Furthermore, step two also includes a multi-source data fusion step: using the analytic hierarchy process (AHP) to determine the weights of different sensor data, and then performing a weighted summation of the normalized data to obtain a comprehensive decision index.

[0021] Multi-source data fusion specifically refers to the fusion process of data collected by the loader from multiple sources such as LiDAR, vision sensors, and pressure sensors. This data fusion process can be represented by the following formula. Let the data collected from n sensors be x1, x2, ..., x... n The data y after normalization i The calculation method for (i = 1, 2, ..., n) is as follows:

[0022]

[0023] Where min(x) and max(x) are the data x1, x2, ..., x3 respectively. n The minimum and maximum values ​​in the data. The normalized data are then weighted according to certain criteria. By integrating the results, we obtain the decision index Z:

[0024]

[0025] weight w i The determination of the analytic hierarchy process (AHP) was carried out using the following steps:

[0026] 1) Constructing the judgment matrix: The importance of each sensor's data is compared pairwise to obtain the judgment matrix A = (a ij ) n×n ,where a ij This indicates the importance of the i-th data point relative to the j-th data point, and n represents the number of sensors.

[0027] 2) Calculate the weight vector: by calculating the largest eigenvalue λ of the judgment matrix A. max The corresponding feature vector W is used to normalize the feature vector, resulting in the weight vector w = (w1, w2, ..., w2) for each data point. n ) T .

[0028] 3) Consistency check: calculate the consistency index and find the corresponding average random consistency index RI, when the consistency ratio is less than 0.1, the judgment matrix is considered to have satisfactory consistency, and the weight vector is valid.

[0029] Further, in step two, the fuzzy logic converts numerical characteristics such as stockpile height, material density, and surface flatness into fuzzy language variable sets such as "high / medium / low", "compact / moderate / loose", and "good / average / poor" through pre-set membership functions.

[0030] Fuzzy logic and uncertainty processing, specifically:

[0031] Take "stockpile height" as an example to build a fuzzy logic model. Let the domain of stockpile height be X = [0, H max ] (H max is the maximum stockpile height that can actually occur), define three fuzzy sets "high stockpile height", "medium stockpile height", and "low stockpile height", and their membership functions are respectively:

[0032] The membership function μ 高 (x) of "high stockpile height" is:

[0033]

[0034] The membership function μ 中 (x) of "medium stockpile height" is:

[0035]

[0036] The membership function μ 低 (x) of "low stockpile height" is:

[0037]

[0038] For other characteristics such as material density, surface flatness, etc., fuzzy sets and membership functions are also defined in a similar manner.

[0039] Further, in step three, the deep reinforcement learning model is a deep Q network (DQN), which includes state space S, action space A, reward function R, and policy function π.

[0040] The state space S is composed of state input and comprehensive decision index Z, specifically including the position coordinates (x, y) of the loader, the inclination angle θ, the current load F of the bucket, the stockpile height h, the material density ρ, etc., i.e. S = {Z, x, y, θ, F, h, ρ, …}.

[0041] The action space A is composed of the shovel action instructions, and specifically includes the shovel path and force that the loader can perform, such as the advancing speed v of the bucket, the lifting height Ah, the shovel angle a, etc., and can be expressed as A = {v, Ah, a, …}.

[0042] The reward function R is set according to the operation effect, with the target shovel amount as the core target, and considering factors such as operation time. It includes positive rewards based on the ratio of the actual shovel amount to the target shovel amount, positive rewards based on operation time efficiency or overtime penalties, and negative penalties based on device collisions or abnormalities.

[0043] The specific rules are as follows:

[0044] 1) Successfully shovel a certain amount of material, give a reward according to the ratio of the actual shovel amount m to the target shovel amount m0, reward value (k1 is the reward coefficient).

[0045] 2) For each unit of time saved in operation time, give a positive reward r2 = k2 (k2 is the time reward coefficient); if the specified time threshold is exceeded, give an overtime penalty r'2 = -k'2 (k'2 is the time penalty coefficient).

[0046] 3) If a collision occurs or the device appears abnormal, give a larger penalty r4 = -k4 (k4 is the larger penalty coefficient).

[0047] The final reward value R = r1 + r2 + r3 + r4 or R = r1 + r2' + r3 + r4.

[0048] The strategy function p, the agent performs an action a e A in the state s e S, obtains a reward r = R(s, a) according to the reward function, and moves to a new state s', and updates the strategy function through the Bellman equation:

[0049]

[0050] where g is the discount factor, 0 < g < 1, and Q(s, a) is the state-action value function. The deep Q network (DQN) is used to realize the learning of the strategy function, and a neural network is constructed to input the state s and output the Q values corresponding to each action. By continuously training the neural network (using the mean square error loss function and updating the network parameters through the back propagation algorithm), the strategy function is optimized, so that the agent can learn the optimal shovel strategy.

[0051] Further, in step two, the model outputs the optimal shovel action instruction, which specifically includes: selecting an action according to the exploration-exploitation balance strategy, randomly selecting an action with a preset probability e, and selecting the action with the maximum Q value with a probability of 1-e; as the training progresses, e is gradually reduced, so that the loader tends to select the optimal action.

[0052] Scheme 2: A loader intelligent control system, comprising:

[0053] a data acquisition module for performing multi-source state perception in step one of scheme 1;

[0054] a data processing module for performing data processing and feature modeling in step two of scheme 1;

[0055] a decision module with a deep reinforcement learning model for performing intelligent decision and optimization in step three of scheme 1;

[0056] a control execution module for performing action execution and feedback in step four of scheme 1.

[0057] The beneficial effects of the present application are:

[0058] (1) This scheme integrates laser radar, vision sensor, pressure sensor and other multi-source data to comprehensively and accurately perceive the characteristics of the material pile and the working state of the loader. Compared with traditional single data source decision, the completeness and reliability of the decision basis are greatly improved, for example, the internal density distribution difference of the material pile can be accurately identified to provide more detailed information for the shovel path planning. And the deep Q network reinforcement learning algorithm is introduced, so that the loader can learn and optimize the shovel decision based on historical operation data and real-time working conditions. It breaks the limitations of traditional fixed rule decision, such as quickly adjusting the shovel angle and speed in complex and variable material pile working conditions to realize dynamic adaptive decision.

[0059] (2) This scheme constructs a reinforcement learning decision model based on deep Q network, carefully designs state space, action space and reward function. The state space covers material pile height, material density, loader position and attitude and other multi-dimensional information; the action space includes shovel lifting, forward and backward, rotation and other operations; the reward function is closely related to the shovel amount, operation time and energy consumption to realize efficient and accurate decision. Through the deep Q network reinforcement learning algorithm introduced in this scheme, the loader can break free from the constraints of fixed rules, learn and continuously optimize the shovel strategy according to real-time working conditions and historical experience, and has strong adaptive ability to complex and variable working environment.

[0060] (3) This scheme can realize accurate control of the target shovel amount through dynamic optimization of the shovel path and force by the intelligent decision model, effectively reducing the underload or overload situation, thereby improving the single operation efficiency and the smoothness of the overall operation process.

[0061] Additional advantages, objects, and features of the application will be apparent to those skilled in the art upon examination of the following detailed description, it being understood that each of the foregoing general statements are true of the particular embodiments of the application. The objects and other advantages of the application can be better understood from the following description in connection with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to make the objects, technical solutions and advantages of the present application clearer, the preferred embodiments of the present application will be described in detail below with reference to the drawings, in which:

[0063] Figure 1 The overall flowchart of the loader target shovel volume dynamic decision method provided by the embodiment of the present application. DETAILED DESCRIPTION

[0064] The other advantages and effects of the present application can be easily understood by those skilled in the art from the disclosure of the present specification. The present application can also be implemented or applied by different specific embodiments, and the details in the present specification can be modified or changed in different views and applications without departing from the spirit of the present application. It should be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0065] The drawings are only used for exemplary illustration, and the representation is only a schematic diagram, not a physical diagram, and cannot be understood as a limitation on the present application; in order to better illustrate the embodiments of the present application, some components in the drawings are omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some known structures and their descriptions in the drawings can be omitted.

[0066] The same or similar reference numerals in the drawings of the embodiments of the present application correspond to the same or similar components; in the description of the present application, it should be understood that if the terms "upper", "lower", "left", "right", "front", "back" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore the terms describing the positional relationship in the drawings are only used for exemplary illustration, and cannot be understood as a limitation on the present application, for those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0067] Embodiment 1:

[0068] The embodiment of the application provides a loader shovel quantity dynamic decision method based on multi-source data fusion and reinforcement learning, and specifically comprises the following steps.

[0069] Step 1: data acquisition and preprocessing

[0070] After the loader starts the operation, multiple types of sensors such as laser radars, visual sensors and pressure sensors work cooperatively to collect relevant data of the stockpile and the loader in real time, and the specific collection contents are shown in Table 1.

[0071] Table 1 data collection contents

[0072]

[0073] The collected data often contains noise interference and dimension difference, and the following processes are used for preprocessing:

[0074] (1) filtering processing:

[0075] For the profile coordinate data collected by the laser radar, a median filtering algorithm is used to remove abnormal noise points. For a sequence x1, x2, …, xn containing n data points, it is arranged in ascending order, and the output y of the median filtering is the data value at the middle position after sorting. If n is odd, n if n is even, i

[0076] For the bucket force data collected by the pressure sensor, a Kalman filtering algorithm is used for processing. According to the dynamics model of the loader and the measurement noise model of the sensor, a Kalman filtering equation is established, and through the prediction and update steps, the real force condition of the bucket is estimated in real time, and the noise interference is effectively suppressed.

[0077] (2) normalization processing:

[0078] The minimum-maximum normalization method is used to map the data of different dimensions to the interval [0, 1]. Taking the stockpile height data h as an example, if the minimum value of the stockpile height data is h min , and the maximum value is h max , then the normalized data is:

[0079]

[0080] For other data such as the bucket force F and the inclination angle θ of the loader, normalization processing is also performed in the same way.

[0081] Step 2: feature mapping and fuzzification

[0082] ​​Based on the pre-processed data, the multi-dimensional features of the stockpile are mapped to fuzzy language variables, and appropriate membership functions are designed for each feature. The membership of each feature value in different fuzzy language variables is calculated, and a fuzzy decision matrix is constructed. Taking the stockpile height, material density, and surface flatness as examples, the fuzzy processing rules are shown in Table 2:

[0083] Table 2 Fuzzy processing rules for stockpile features

[0084]

[0085] Assuming that after calculation, the membership of the stockpile height for "low", "medium", and "high" is 0.1, 0.3, and 0.6 respectively, the membership of the material density for "loose", "moderate", and "compact" is 0.2, 0.5, and 0.3 respectively, and the membership of the surface flatness for "poor", "average", and "good" is 0.3, 0.4, and 0.3 respectively, then the fuzzy decision matrix M is:

[0086]

[0087] Step 3: Intelligent decision and optimization

[0088] Based on the fuzzy decision matrix, the reinforcement learning algorithm is used for intelligent decision-making, and the specific process is as follows:

[0089] (1) Initialization: Initialize the parameters of the deep Q network, set the learning rate α, discount factor γ, exploration rate ∈, and other hyperparameters, and set the initial state S0 as the state when the loader starts. The recommended value range for hyperparameters: learning rate α between 0.001-0.1, discount factor γ between 0.8-0.99, and exploration rate ∈ initial value set to 0.9, and gradually reduced to 0.1 during training with a certain decay rate.

[0090] (2) State input: Combine the fuzzy decision matrix and other state parameters of the loader into state S t , input into the deep Q network, and the network outputs the Q value corresponding to each action, i.e. Q(s t ,a). The network structure can use a 3-layer fully connected neural network, the number of input layer nodes is determined according to the number of state parameters, the number of hidden layer nodes is set to 128 and 64 respectively, and the number of output layer nodes is the same as the dimension of the action space.

[0091] (3) Action selection: Select action a t according to the exploration-exploitation balance strategy, randomly select action with probability ∈, and select the action with the maximum Q value with probability 1-∈. In the early stage of training, ∈ is larger, making the loader explore different actions more; as the training progresses, ∈ is gradually reduced, making the loader more inclined to choose the optimal action.

[0092] (4) Action Execution: The selected action a t is sent to the loader control system, which controls the loader to perform the corresponding material shoveling operation, such as adjusting the forward speed, lifting height, and shoveling angle of the bucket, etc. The transmission of action instructions uses the industrial Ethernet protocol to ensure that the instructions are quickly and accurately delivered to each actuator of the loader.

[0093] (5) Reward Acquisition: After the loader performs the action, the actual operation effect is evaluated, and the reward value r t is calculated according to the reward function R.

[0094] (6) State Transition: After the loader performs the action, the environment changes, the sensor collects new data, and the state is updated to s t+1 .

[0095] (7) Q Value Update: The target Q value is calculated according to the Bellman equation , and the mean square error loss function is used to update the parameters of the deep Q network through the backpropagation algorithm, so that the Q value output by the network gradually approaches the optimal value.

[0096] (8) Repeat Training: Repeat steps (2) to (7) to continuously train until the loader learns a stable optimal strategy, i.e., the Q value converges or reaches a preset number of training times.

[0097] During decision-making, real-time monitoring of changes in the operating environment is performed, and when the environmental changes exceed a certain threshold (such as a significant change in the shape of the material pile or the appearance of new obstacles), state assessment and decision-making are re-performed.

[0098] Example 2:

[0099] Please refer to Figure 1 , which shows an intelligent control system for a loader that dynamically obtains optimized loader material shoveling decision results. The specific control process steps are broken down as follows:

[0100] (1) Get Instruction (Start):

[0101] Get optimized material shoveling decision results: The system receives the rate instruction for adding material this time from a higher-level optimization algorithm or plan.

[0102] (2) Execute Action:

[0103] Send control instructions through industrial Ethernet using the Modbus-TCP protocol; the control system parses the instructions to drive the actuator to act.

[0104] (3) Safety Work and Process Monitoring:

[0105] If the force on the bucket exceeds the safety threshold, the system triggers an alarm and suspends the operation, and records the abnormal data. If not, the tilt sensor monitors the tilt angle of the tilting machine, and determines whether the tilt angle exceeds 30 degrees. If yes, the system automatically adjusts the bucket posture. If not, the bucket continues to complete the loading operation.

[0106] (4) Result measurement and feedback:

[0107] After the loading process is completed, the weighing sensor measures the actual furnace intake, and then compares it with the target bucket intake to calculate the error. If the error exceeds 5%, the system retrieves multi-source data, analyzes the error using fuzzy logic, adjusts the decision model and algorithm parameters based on the error analysis, updates the reward function weight coefficient, and optimizes the deep Q network structure parameters. If not, the operation data is recorded to the historical database, and the historical data is analyzed regularly to summarize the optimal decision mode.

[0108] Finally, it should be pointed out that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A loader shovel material dynamic decision method based on multi-source data fusion and reinforcement learning, characterized in that, The method specifically comprises: Multi-source state perception: Real-time collection of state data of the stockpile and the loader through at least one sensor mounted on the loader; the sensor includes one or more combinations of a laser radar, a visual sensor, a pressure sensor, an inclination sensor and a position sensor; Data processing and feature modeling: preprocessing of the collected state data, and construction of a feature model describing the working environment; the preprocessing includes filtering processing and normalization processing; the construction of the feature model includes mapping the processed data into fuzzy language variables by using fuzzy logic to construct a fuzzy decision matrix; Intelligent decision and optimization: inputting the fuzzy decision matrix and other state parameters as state inputs into a pre-trained deep reinforcement learning model, and outputting optimal shovel action instructions by the model; the shovel action instructions include at least one of the advancing speed, the lifting height and the digging angle of the bucket; Action execution and feedback: controlling the loader to execute the shovel action instructions, and calculating a reward value according to the difference between the actual shovel amount and the target shovel amount after execution and the working time, and using the reward value to update or train the deep reinforcement learning model online or offline.

2. The loader shovel dynamics decision method of claim 1, wherein, In the multi-source state perception, the state data includes the height, slope, contour coordinates, surface texture, color and shape of the stockpile, and the force on the bucket, material density, inclination angle of the loader and position coordinates of the loader.

3. The loader shovel dynamics decision method of claim 1, wherein, In the data processing and feature modeling, the filtering processing includes using a median filtering algorithm to denoise the contour coordinate data collected by the laser radar, and / or using a Kalman filtering algorithm to denoise the force data on the bucket collected by the pressure sensor.

4. The loader shovel dynamics decision method of claim 1, wherein, In the data processing and feature modeling, it further includes a multi-source data fusion step: using an analytic hierarchy process to determine the weights of different sensor data, and performing weighted summation on the normalized data to obtain a comprehensive decision index.

5. The loader shovel dynamics decision method of claim 1, wherein, In the data processing and feature modeling, the fuzzy logic converts the stockpile height, material density and surface flatness into a fuzzy language variable set of "high / medium / low", "compact / moderate / loose" and "good / average / poor" through a preset membership function.

6. The loader shovel dynamics decision method of claims 1 or 4, wherein, In the intelligent decision and optimization, the deep reinforcement learning model is a deep Q network, including a state space S, an action space A, a reward function R and a policy function π; the state input and the comprehensive decision index constitute the state space S of the deep Q network; the shovel action instructions constitute the action space A of the deep Q network.

7. The loader shovel dynamics decision method of claim 6, wherein, In the action execution and feedback, the reward value is determined by the reward function R, and the reward function at least includes one or more combinations of: Positive reward based on the ratio of the actual shovel amount to the target shovel amount; Positive reward based on the efficiency of the working time or overtime penalty; Negative penalty based on the collision or abnormality of the equipment.

8. The loader shovel dynamics decision method of claim 6, wherein, The policy function π specifically comprises: an agent performs an action a e A in a state s e S, obtains a reward according to a reward function, and moves to a new state s ′ The policy function is updated through a Bellman equation; the learning of the policy function is implemented by using a deep Q network, a neural network is constructed, the state is input, the Q values corresponding to each action are output, a mean square error loss function is used, the network parameters are updated through a back propagation algorithm, and the policy function is optimized, so that the agent can learn an optimal shoveling strategy.

9. The loader shovel dynamics decision method of claim 8, wherein, In the intelligent decision and optimization, the model outputs optimal shovel action instructions, specifically comprising: selecting an action according to an exploration-exploitation balance strategy, randomly selecting an action with a preset probability ∈, and selecting an action with the maximum Q value with a probability of 1-∈; as the training proceeds, gradually reducing ∈, so that the loader is more inclined to select the optimal action.

10. A loader intelligent control system characterized by, Comprise: a data acquisition module, a data processing module, a decision module and a control execution module; The data acquisition module is used for executing the multi-source state perception of claim 1; The data processing module is used for executing the data processing and feature modeling of claim 1; The decision module is used for executing the intelligent decision and optimization of claim 1, and a deep reinforcement learning model is built in the decision module; The control execution module is used for executing the action execution and feedback of claim 1.

Images