A blanking control method based on reinforcement learning

Abstract

The application provides a blanking control method based on reinforcement learning, comprising the following steps: collecting a live image by using a camera; wherein the live image comprises a front image, a side image and a top-down image of a vehicle and a blanking port; extracting a live parameter according to the live image and live equipment information by using a preset instance segmentation model; analyzing the live parameter as a current state parameter, and controlling the position of the vehicle and the size of the blanking port, comprising the following steps: setting a training stage and a use stage; adjusting the position of the vehicle and the size of the blanking port according to different current state parameters in the training stage, repeatedly training the content of the training stage through feedback rewards and penalties until the vehicle can be fully loaded in the shortest time; and controlling the position of the vehicle and the size of the blanking port according to the strategy trained in the training stage in the use stage. The application realizes intelligent control of the whole process, and each stage does not need human intervention and operation.

Description

Technical Field

[0001] This invention relates to the fields of reinforcement learning, intelligent vehicle and cargo control technology, and in particular to a material dropping control method based on reinforcement learning. Background Technology

[0002] Currently, material unloading control at transfer points such as port warehouses mainly relies on the coordinated efforts of operators in the control room and vehicle drivers. Control measures include unloading height, vehicle position, and unloading speed. Further automation is limited to camera-based vehicle position recognition and simple adjustments. Existing technology lacks a unified, intelligent processing system for the entire unloading process; the coordination between unloading and vehicle operation still depends on adjustments and control based on human experience.

[0003] Existing technologies have limitations on the speed of the overall material unloading process, namely, how to ensure that each vehicle can unload and load materials at the fastest speed and with the largest loading and unloading volume. Manual adjustment usually controls the speed and constantly adjusts the vehicle's position to allow for as much loading and unloading as possible. However, excessively fast unloading speeds make it difficult to guarantee that there will be no overflow of materials or insufficient utilization of cargo space. Summary of the Invention

[0004] The purpose of this invention is to at least address one of the aforementioned technical deficiencies.

[0005] Therefore, the purpose of this invention is to propose a material dropping control method based on reinforcement learning.

[0006] To achieve the above objectives, embodiments of the present invention provide a material feeding control method based on reinforcement learning, comprising the following steps:

[0007] Step S1: Use a camera to capture on-site images; wherein, the on-site images include: frontal images, side images and top-view images of the vehicle and the material drop-out port;

[0008] Step S2: Using a preset instance segmentation model, extract field parameters based on the field images and field equipment information; wherein, the field parameters include: vehicle position, material discharge port position, material discharge port speed, and existing material discharge volume in the carriage;

[0009] Step S3 involves analyzing the field parameters as current state parameters to control the vehicle position and the size of the material drop opening, including:

[0010] Step S31: Set up the training phase and the usage phase;

[0011] Step S32: During the training phase, the vehicle position and the size of the material drop opening are adjusted according to different current state parameters. The training phase is repeated through feedback rewards and penalties until the vehicle can be fully loaded in the shortest time.

[0012] Step S33: In the usage phase, the vehicle position and the size of the material drop opening are controlled according to the strategy trained in the training phase; wherein, the trained strategy is a model obtained by using the reward and punishment mechanism in step S32. The input of this model is the on-site image and the current state parameters, and the vehicle position and the size of the material drop opening are adjusted according to a preset time interval.

[0013] Furthermore, in step S2, the vehicle location information and the material drop port location information are extracted, including the following steps: using an instance segmentation model to accurately identify the vehicle compartment position and the material drop port position in the image, forming the coordinate information Loc of the vehicle and the material drop port.

[0014] Furthermore, an instance segmentation model is used to identify the side image, detecting the coordinates (X, Y) of the material discharge port in the image. L ,Y L ) and the position (Xc, Yc) in the vehicle's side view image;

[0015] The instance segmentation model was used to identify the top view image, and the position information (L) of the material discharge port on the horizontal plane was detected. l ,R l The vehicle's position information (Lc, Rc) on the horizontal plane;

[0016] The above coordinate information is used to form coordinate information.

[0017] Loc = [(X L ,Y L );(Xc,Yc);(L l ,R l );(Lc,Rc)].

[0018] Furthermore, the instance segmentation model adopts the Mask-rcnn model, the Faster-r-cnn model, or the segmentanything model.

[0019] Furthermore, in step S2, extracting the existing material discharge volume inside the carriage includes the following steps:

[0020] The volume currently occupied inside the container is estimated by dividing the carriage into four parts: front, back, left, and right. The volume of each part is estimated based on the image information, resulting in the volume information Val = (Val F Val B Val L Val R), where Val F ValB is the estimated volume of the front section, ValL is the estimated volume of the rear section, ValR is the estimated volume of the left section, and ValR is the estimated volume of the right section.

[0021] Furthermore, obtain the estimated front volume value Val. F and rear volume estimate Val B It includes the following steps:

[0022] The carriage image obtained by position detection is extracted in the horizontal direction;

[0023] The carriage was divided into two parts, front and rear, and processed separately.

[0024] For each part, a model based on the Vision Transformer model is trained for a similar classification task.

[0025] During the training phase, the images of the first and second halves of the carriage are encoded by position, and the amount of material already discharged by the device at this time is used as a weighting value for the embedding layer.

[0026] During the usage phase, the segmented image is input into the trained Vision Transformer model to obtain the front volume estimate Val. F and rear volume estimate Val B .

[0027] Furthermore, obtain the estimated volume value Val for the left side. L and the estimated volume of the right side Val R It includes the following steps:

[0028] The carriage image obtained by position detection is extracted in the vertical direction;

[0029] The carriage was divided into left and right sections, and each section was processed separately.

[0030] For each part, a model based on the Vision Transformer model is trained for a similar classification task.

[0031] During the training phase, the left and right halves of the carriage images are encoded by position, and the amount of material already dispensed by the device at this time is used as a weighting value for the embedding layer.

[0032] During the usage phase, the segmented image is input into the trained Vision Transformer model to obtain the estimated left volume value Val. L and the estimated volume of the right side Val R .

[0033] Furthermore, in step S3,

[0034] During the training phase, the penalty is maximized if the cargo box overflows, and the normal reward is given if the cargo box is not full. The reward is set to decrease over time to ensure that the marginal benefit decreases over time.

[0035] During the training phase, to ensure that the training objective takes the least amount of time, the sum of the rewards at each moment is divided by the total time taken, and the resulting value is used as an inverse coefficient. Based on the inverse coefficient, the training objective is set to maximize the final reward.

[0036] Furthermore, in step S3, the training phase includes the following steps:

[0037] Obtain the location information in the image and the size of the discharge port; the current volume is used as the current state at time t in reinforcement learning.

[0038] S t =(Val (t) Loc (t) );

[0039] Among them, S t This represents the current state at time t; Val (t) Loc represents the volume parameter at time t. (t) This refers to the current position information at time t.

[0040] Give operation action A t = (ΔLoc, ΔV, ...), where ΔLoc is the coordinate of the vehicle's position change and ΔV is the adjustment speed of the material drop outlet;

[0041] At time t+1, the current state has changed to S. t+1 =(Val (t+1) Loc (t+1) ), receive feedback reward R t+1 The feedback reward condition is set such that if the cargo box overflows, the penalty is maximized; if the cargo box is not full, a normal reward is given. The reward is set to decrease over time to ensure that marginal revenue decreases over time; where S t+1 The state at time t+1; Val (t+1) Let Loc be the volume parameter at time t+1. (t+1) This refers to the location information at time t+1;

[0042] Repeat the above process until the vehicle is fully loaded, and denote this moment as T. At this point, calculate the overall feedback reward.

[0043] G = (R0 + R1 + ... + R) T ) / T;

[0044] Where R0 represents the reward or penalty for the operation at time 0; R1 represents the reward or penalty for the operation at time 1; R T G represents the reward or penalty for the action at time T; G ​​is the overall feedback reward, i.e., the total reward obtained. G is inversely proportional to time T, and the longer the time, the lower the reward is consumed.

[0045] To ensure that the goal is to minimize the time consumption, the sum of the rewards at each time step is divided by the total time consumption T as an inverse coefficient. The training objective is set to maximize the final reward G.

[0046] Furthermore, in step S3, during the usage phase, the input state S will be automatically adjusted according to the trained strategy. t Perform the corresponding action A t This achieves the same effect as the training objective, which is to fill the vehicle as much as possible in the shortest amount of time.

[0047] The material feeding control method based on reinforcement learning according to embodiments of the present invention has the following beneficial effects:

[0048] (1) Use reinforcement learning to automate the overall material feeding process, reduce reliance on manual labor, and maximize process efficiency.

[0049] (2) The entire material feeding system is parameterized, and the process is integrated into the overall park's automated scheduling in the form of parameters, providing basic conditions for further realizing automated scheduling within the park.

[0050] (3) Achieve intelligent control of the entire process, and no human intervention or operation is required at each stage.

[0051] (4) High replicability: After training for multiple scenarios, this method can be replicated and used in scenarios in other regions. By leveraging the replicability of the overall model, the learning cost for new locations is reduced; adaptation to new locations can be achieved simply by copying the model.

[0052] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0053] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0054] Figure 1 This is a flowchart of a reinforcement learning-based material feeding control method according to an embodiment of the present invention;

[0055] Figure 2This is a schematic diagram of a reinforcement learning-based material feeding control method according to an embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram illustrating the identification of the vehicle compartment position and the material unloading port position of a vehicle according to an embodiment of the present invention.

[0057] Figure 4 This is a schematic diagram showing that the carriage according to an embodiment of the present invention is divided into four parts: front, back, left, and right.

[0058] Figure 5 This is a schematic diagram of the training phase according to an embodiment of the present invention. Detailed Implementation

[0059] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0060] This invention proposes a material feeding control method based on reinforcement learning. This method learns the control flow of all stages based on reinforcement learning content, thereby forming an overall intelligent process control. By using reinforcement learning to treat vehicle position adjustment and material feeding port size control as actions, and the current vehicle position and used volume as the current state, the method automatically adjusts based on the strategy obtained from reinforcement learning.

[0061] like Figure 1 and Figure 2 As shown, the reinforcement learning-based material feeding control method of this invention includes the following steps:

[0062] Step S1: Use a camera to capture images of the scene.

[0063] Specifically, the on-site equipment in the port warehouse includes vehicles, material unloading points, and surveillance cameras.

[0064] The cameras acquire various information from the site, capturing images of the warehouse. These images include front, side, and top views of the vehicles and material handling chute. In other words, the cameras capture images of the vehicles and material handling chute from the front, side, and above.

[0065] Step S2 involves using a pre-defined instance segmentation model to extract on-site parameters based on the on-site images and equipment information. The instance segmentation model accurately obtains the location and edge information of the vehicle in the monitoring image.

[0066] Specifically, in this step, the on-site images and equipment information collected in step S1 are analyzed. The on-site equipment information includes parameters such as the size of the material discharge port. After analysis, the on-site parameters are extracted. These parameters include: vehicle position, material discharge port position, material discharge port speed, and the existing material volume in the vehicle compartment. The instance segmentation model obtains the vehicle position and material discharge port position information, and the on-site material discharge port speed can be obtained through equipment parameter transmission. All of these are required as input to the subsequent reinforcement learning model.

[0067] (1) Extract vehicle location information and material discharge port location information, including the following steps:

[0068] An instance segmentation model is used to accurately identify the vehicle's cargo compartment and material unloading port position in the image, forming the coordinate information (Loc) of the vehicle and the material unloading port. Figure 3 Position A corresponds to the material drop port, and position B corresponds to the vehicle position.

[0069] The side image is identified using an instance segmentation model. Specifically, the side camera is used to detect the coordinates (X, Y, Z) of the material discharge port in the image using the instance segmentation model. L ,Y L ) and the position (Xc, Yc) in the vehicle's side view image.

[0070] An instance segmentation model is used to identify the top-view image. That is, the top-view camera uses the instance segmentation model to detect the position information of the material inlet on the horizontal plane (L). l ,R l ) and the vehicle's position information (Lc, Rc) on the horizontal plane.

[0071] The above coordinate information is used to form coordinate information.

[0072] Loc = [(X L ,Y L );(Xc,Yc);(L l ,R l );(Lc,Rc)].

[0073] In embodiments of the present invention, the instance segmentation model may employ the Mask-R-CNN model, the Faster-R-CNN model, or the Segment Anything model. It should be noted that the instance segmentation model is not limited to the examples described above; other image segmentation models may also be used, which will not be elaborated upon here.

[0074] For instance segmentation model training, high-precision location detection can be achieved by fine-tuning the existing segmentation model. In practice, the intermediate results of this detection can be used as input to downstream control modules, and this information can also be directly displayed as intermediate output.

[0075] (2) Extract the existing material discharge volume inside the car, including the following steps:

[0076] like Figure 4 As shown, the currently occupied volume inside the container is estimated. The carriage is divided into four parts: front, back, left, and right. The volume of each part is estimated based on the image information, resulting in the volume information Val = (Val F Val B Val L Val R Among them, Val F ValB is the estimated volume of the front section, ValL is the estimated volume of the rear section, ValR is the estimated volume of the left section, and ValR is the estimated volume of the right section.

[0077] (2.1) Obtain the estimated front volume Val F and rear volume estimate Val B It includes the following steps:

[0078] First, the carriage image obtained by position detection is extracted in the horizontal direction.

[0079] Next, the carriage is divided into front and rear sections, and processed separately. Further subdivisions can be made, such as three or more sections, using the same method.

[0080] Secondly, for each part, a model based on the Vision Transformer (Vit) model is trained for a similar classification task.

[0081] Secondly, during the training phase, the images of the first and second halves of the carriage are encoded by position, and the amount of material already dispensed by the device at this time is used as a weighting value for the embedding layer.

[0082] For example: Original image paths: E = [E1, E2, E3, ... E 1024 The weighted value for material unloading is: a = already unloaded material / actual load capacity of the car. The final input to the Encoder is: Input = a * E.

[0083] The training classification target value is: the percentage of this part of the carriage that has been used, 0%, 20%, 40%, 60%, 80%, 100%. Finally, in the usage phase, after inputting the segmented image portion into the trained Vision Transformer model, the front volume estimate Val is obtained. F and rear volume estimate Val B .

[0084] (2.2) Obtain the estimated volume value Val of the left side L and the estimated volume of the right side Val R It includes the following steps:

[0085] First, the carriage image obtained by position detection is extracted in the vertical direction.

[0086] Next, divide the carriage into left and right sections and process them separately. Further division is possible, with three or more sections, using the same method.

[0087] Secondly, for each part, a model based on the Vision Transformer model is trained for a similar classification task.

[0088] Secondly, during the training phase, the left and right halves of the carriage images are encoded by position, and the amount of material already dispensed by the device at this time is used as a weighting value for the embedding layer.

[0089] Finally, in the usage phase, the segmented image is input into the trained Vision Transformer model to obtain the estimated left volume value Val. L and the estimated volume of the right side Val R .

[0090] In summary, through steps (2.1) and (2.2), the final output control information is: Val = (Val F Val B Val L Val R ).

[0091] Step S3 involves analyzing the on-site parameters as current state parameters to control the vehicle position and the size of the material drop opening, including:

[0092] Step S31: Set up the training phase and the usage phase.

[0093] Step S32: During the training phase, adjust the vehicle position and the size of the material drop opening according to different current state parameters, and repeat the training phase content through feedback rewards and penalties until the vehicle can be fully loaded in the shortest time.

[0094] Specifically, during the training phase, the penalty is maximized if the cargo box overflows; otherwise, a normal reward is given. The reward is set to decrease over time to ensure that marginal returns decrease with time. To ensure that the training objective is to minimize the training time, the sum of the rewards at each time step is divided by the total time, and the resulting value is used as an inverse coefficient. Based on this inverse coefficient, the training objective is set to maximize the final reward.

[0095] The following is combined Figure 5 The steps of the training phase are explained.

[0096] First, obtain the location information in the image and the size of the material inlet. The current volume is used as the current state at time t in reinforcement learning.

[0097] S t =(Val (t) Loc (t) );

[0098] Among them, S t This represents the current state at time t; Val (t) Loc represents the volume parameter at time t. (t) This represents the current position information at time t.

[0099] Then, give operation action A. t = (ΔLoc, ΔV, ...), where ΔLoc is the coordinate of the vehicle's position change and ΔV is the adjustment speed of the material drop outlet.

[0100] At time t+1, the current state has changed to S. t+1 =(Val (t+1) Loc (t+1) ), receive feedback reward R t+1 The feedback reward condition is set such that if the cargo box overflows, the penalty is maximized; if the cargo box is not full, a normal reward is given. The reward is set to decrease over time to ensure that marginal revenue decreases with increasing time. Where S... t+1 The state at time t+1; Val (t+1) Let Loc be the volume parameter at time t+1. (t+1) This refers to the location information at time t+1.

[0101] Repeat the above process until the vehicle is fully loaded, and denote this moment as T. At this point, calculate the overall feedback reward.

[0102] G = (R0 + R1 + ... + R) T ) / T.

[0103] Where R0 represents the reward or penalty for the operation at time 0; R1 represents the reward or penalty for the operation at time 1; and so on, R...T G is the reward or penalty for the operation at time T; G ​​is the overall feedback reward, i.e. the total reward obtained. G is inversely proportional to time T. The longer the time, the lower the reward is consumed, because the goal is to fill the vehicle in the shortest possible time.

[0104] To ensure that the goal is to minimize the time consumption, the sum of the rewards at each time step is divided by the total time consumption T as an inverse coefficient. The training objective is set to maximize the final reward G.

[0105] Specifically, G is the objective function. During model training, for example, in the first round of training, the total reward (the sum of all R values) is 10, and T = 5. Therefore, the objective function G = 2, and the resulting model is Model1. In the second round of training, the total reward (the sum of all R values) is 16, and T = 10. Therefore, the objective function G = 1.6, and subsequent models will continue to be trained on Model2. The training content of the model follows the basic training method for deep learning models, with the objective function and model parameters trained through forward and backward propagation.

[0106] Step S33: During the usage phase, the vehicle position and the size of the material drop opening are controlled according to the strategy trained during the training phase.

[0107] In this step, the trained strategy refers to obtaining a model using the reward and penalty mechanism in step S32. This model is equivalent to taking an image and some parameters as input and adjusting the vehicle position and the size (speed) of the material drop chute at regular time intervals. Similar to the training process of AI playing chess, the goal during training is to win in the end. At each move, a reward or penalty is given to optimize the final result. When using the model, the current information is input, and the model adjusts its next movement strategy—that is, the vehicle position and the size (speed) of the material drop chute—based on the training results.

[0108] During the usage phase, the model will automatically perform the corresponding action At for the input state St based on the trained strategy, thereby achieving the same effect as the training objective: to fill the vehicle as much as possible in the shortest time. Once the model training is complete, these parameters will no longer change and will remain fixed values. During inference (usage), all parameters are in a fixed state. There will still be a reward; the model itself can determine that the current operation (providing corresponding instructions, such as moving the vehicle left or right, or accelerating or decelerating the unloading) is the optimal operation.

[0109] During the usage phase, a strategy model is used to obtain corresponding control commands. For example, operations such as the distance the material discharge port moves forward or backward, and increasing or decreasing the material discharge speed are directly transmitted to the equipment control system for adjustment.

[0110] The material feeding control method based on reinforcement learning according to embodiments of the present invention has the following beneficial effects:

[0111] (1) Use reinforcement learning to automate the overall material feeding process, reduce reliance on manual labor, and maximize process efficiency.

[0112] (2) The entire material feeding system is parameterized, and the process is integrated into the overall park's automated scheduling in the form of parameters, providing basic conditions for further realizing automated scheduling within the park.

[0113] (3) Achieve intelligent control of the entire process, and no human intervention or operation is required at each stage.

[0114] (4) High replicability: After training for multiple scenarios, this method can be replicated and used in scenarios in other regions. By leveraging the replicability of the overall model, the learning cost for new locations is reduced; adaptation to new locations can be achieved simply by copying the model.

[0115] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0116] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A material feeding control method based on reinforcement learning, characterized in that, Includes the following steps: Step S1: Use a camera to capture on-site images; wherein, the on-site images include: frontal images, side images and top-view images of the vehicle and the material drop-out port; Step S2: Using a preset instance segmentation model, extract field parameters based on the field images and field equipment information; wherein, the field parameters include: vehicle position, material discharge port position, material discharge port speed, and existing material discharge volume in the carriage; Extracting vehicle location information and material drop port location information includes the following steps: using an instance segmentation model to accurately identify the vehicle compartment position and material drop port position in the image, forming the coordinate information Loc of the vehicle and the material drop port; Extracting the existing material discharge volume inside the car body includes the following steps: The volume currently occupied inside the container is estimated by dividing the carriage into four parts: front, back, left, and right. The volume of each part is estimated based on the image information, resulting in the volume information Val = (Val F Val B Val L Val R ), of which Val F For the estimated front volume, Val B For the estimated rear volume, Val L Estimated volume for the left side, Val R This is an estimated volume for the right side; Step S3 involves analyzing the field parameters as current state parameters to control the vehicle position and the size of the material drop opening, including: Step S31: Set up the training phase and the usage phase; Step S32: During the training phase, the vehicle position and the size of the material drop opening are adjusted according to different current state parameters. The training phase is repeated through feedback rewards and penalties until the vehicle can be fully loaded in the shortest time. During the training phase, the penalty is maximized if the cargo box overflows, and the normal reward is given if the cargo box is not full. The reward is set to decrease as time increases to ensure that the marginal benefit decreases with time. During the training phase, in order to ensure that the training objective takes the least amount of time, the sum of the rewards at each time point is divided by the total time, and the resulting value is used as an inverse coefficient. Based on the inverse coefficient, the training objective is set to maximize the final reward. The training phase consists of the following steps: Obtain the location information in the image and the size of the discharge port; the current volume is used as the current state at time t in reinforcement learning. S t =(Wave (t) ,Place (t) ); Among them, S t This represents the current state at time t; Val (t) Loc represents the volume parameter at time t. (t) The position at time t information; Give operation action A t = (ΔLoc, ΔV), where ΔLoc is the coordinate of the vehicle's position change and ΔV is the adjustment speed of the material drop outlet; At time t+1, the current state has changed to S. t+1 =(Val (t+1) Loc (t+1) ), receive feedback reward R t+1 The feedback reward condition is set such that if the cargo box overflows, the penalty is maximized; if the cargo box is not full, a normal reward is given. The reward is set to decrease over time to ensure that marginal revenue decreases over time; where S t+1 The state at time t+1; Val (t+1) Let Loc be the volume parameter at time t+1. (t+1) This refers to the location information at time t+1; Repeat the above process until the vehicle is fully loaded, and denote this moment as T. At this point, calculate the overall feedback reward G = (R0 + R1 + ... + R T ) / T; Where R0 represents the reward or penalty for the operation at time 0; R1 represents the reward or penalty for the operation at time 1; R T G represents the reward or penalty for the action at time T; G ​​is the overall feedback reward, i.e., the total reward obtained. G is inversely proportional to time T, and the longer the time, the lower the reward is consumed. To ensure that the goal is to minimize the time consumption, the sum of the rewards at each time step is divided by the total time consumption T as an inverse coefficient. The training objective is set to maximize the final reward G. Step S33: In the usage phase, the vehicle position and the size of the material drop opening are controlled according to the strategy trained in the training phase; wherein, the trained strategy is a model obtained by using the reward and punishment mechanism in step S32. The input of this model is the on-site image and the current state parameters, and the vehicle position and the size of the material drop opening are adjusted according to a preset time interval.

2. The material feeding control method based on reinforcement learning as described in claim 1, characterized in that, The side image was identified using an instance segmentation model, and the coordinates (X, Y) of the material discharge port in the image were detected. L ,Y L ) and the position (Xc, Yc) in the vehicle's side view image; The instance segmentation model was used to identify the top view image, and the position information of the material discharge port on the horizontal plane (L) was detected. l ,R l ) and the vehicle's position information (Lc,Rc) on the horizontal plane; The above coordinate information is used to form coordinate information. Loc=[(X L ,Y L );(Xc,Yc);(L l ,R l );(Lc,Rc)]。 3. The material feeding control method based on reinforcement learning as described in claim 1 or 2, characterized in that, The instance segmentation model uses the Mask-rcnn model, the Faster-r-cnn model, or the segment anything model.

4. The material feeding control method based on reinforcement learning as described in claim 1, characterized in that, Obtain the estimated front volume value Val F and rear volume estimate Val B It includes the following steps: The carriage image obtained by position detection is extracted in the horizontal direction; The carriage was divided into two parts, front and rear, and processed separately. For each part, a model based on the Vision Transformer model is trained for a similar classification task. During the training phase, the images of the first and second halves of the carriage are encoded by position, and the amount of material already discharged by the device at this time is used as a weighting value for the embedding layer. During the usage phase, the segmented image is input into the trained Vision Transformer model to obtain the front volume estimate Val. F and rear volume estimate Val B .

5. The material feeding control method based on reinforcement learning as described in claim 1, characterized in that, Obtain the estimated volume value Val on the left side L and the estimated volume of the right side Val R It includes the following steps: The carriage image obtained by position detection is extracted in the vertical direction; The carriage was divided into left and right sections, and each section was processed separately. For each part, a model based on the Vision Transformer model is trained for a similar classification task. During the training phase, the left and right halves of the carriage images are encoded by position, and the amount of material already dispensed by the device at this time is used as a weighting value for the embedding layer. During the usage phase, the segmented image is input into the trained Vision Transformer model to obtain the estimated left volume value Val. L and the estimated volume of the right side Val R .

6. The material feeding control method based on reinforcement learning as described in claim 1, characterized in that, In step S3, during the usage phase, the input state S will be automatically adjusted according to the trained strategy. t Perform the corresponding action A t This achieves the same effect as the training objective, which is to fill the vehicle as much as possible in the shortest amount of time.

Images