Subway foundation pit excavation equipment based on multi-joint linkage and bionic training method thereof

Abstract

The invention discloses metro foundation pit excavation equipment based on multi-joint linkage and a bionic training method of the metro foundation pit excavation equipment, and relates to the field of metro foundation pit excavation equipment. The equipment comprises a rack, a control cabin, a transportation device, a large arm, a small arm, a bucket, a large arm steering wheel, a small arm hydraulic motor steering wheel, an ultrasonic radar device, a laser radar device, a high-definition camera, a central controller and a crawler walking device. And in combination with multi-sensor fusion data, the success rate of operation obstacle avoidance in a complex, narrow and closed space can be greatly improved.

Description

Technical Field

[0001] This invention relates to the field of subway foundation excavation equipment technology, and in particular to a subway foundation pit excavation equipment based on multi-joint linkage and its biomimetic training method. Background Technology

[0002] In recent years, the scale of subway construction in my country has continued to expand. However, due to regional economic differences and the level of technology and equipment, the open-cut technology system, represented by the cut-and-cover method, faces multiple challenges in subway foundation pit excavation projects, such as complex geological conditions, narrow construction space and high environmental sensitivity.

[0003] Traditional excavation equipment uses a single rotating robotic arm structure, which suffers from insufficient flexibility, low coordination efficiency, and unsuitability for confined environments. Especially in enclosed spaces with multiple pillars, traditional robotic arms are prone to collisions with their surroundings and cannot achieve small-range rotation. Furthermore, current construction methods heavily rely on manual labor, and insufficient mechanization creates a conflict between efficiency and safety: on the one hand, manual excavation in confined spaces poses high safety risks, and the harsh working environment continuously threatens worker health; on the other hand, traditional processes struggle to meet the precision requirements under complex geological conditions, and construction errors can easily trigger a chain reaction.

[0004] Therefore, there is an urgent need to develop a new type of multi-joint linkage foundation pit excavation equipment with flexible robotic arms that can easily avoid obstacles, require little construction space, are highly efficient, and have biomimetic learning capabilities. Summary of the Invention

[0005] In view of this, the present invention provides a subway foundation pit excavation equipment based on multi-joint linkage and its biomimetic training method.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A subway foundation pit excavation equipment based on multi-joint linkage includes: a frame rotatably mounted on a crawler walking device; a control cabin and a boom steering wheel fixedly mounted at the front end of the frame; a boom rotatably connected to the boom steering wheel; a forearm hydraulic motor steering wheel rotatably connected to the boom; a forearm rotatably connected to the forearm hydraulic motor steering wheel; a bucket rotatably connected to the forearm; and a transport device connected to the frame; ultrasonic radar devices are installed on the bottom front side of the control cabin, the bottom right side of the frame, the outer left side of the frame, and the outer right side of the frame; a laser radar device is installed on the upper surface of the control cabin; high-definition camera devices are installed on the front, left, right, rear left, and rear right sides of the frame; the boom has a rotation range of 60° to the left and right; the forearm hydraulic motor steering wheel enables the forearm to have a rotation range of 90° to the left and right; the high-definition camera device, ultrasonic radar device, and laser radar device are communicatively connected to the central controller in the control cabin; the boom (4), forearm hydraulic motor steering wheel (5), and forearm (6) form a series robotic arm, upgrading the motion model to:

[0008] T=T1(θ1)T2(θ2)T3(θ3)T4(θ4)

[0009] Wherein: T4(θ4) is the transformation matrix of the boom slewing mechanism, T1(θ1) is the transformation matrix of the bucket slewing mechanism, T2(θ2) is the transformation matrix of the boom and arm slewing mechanisms, and T3(θ3) is the transformation matrix of the arm slewing mechanism; these are used to improve the reachable working space and obstacle avoidance capability of the end effector, thereby increasing the overall excavation area of ​​the equipment by 30% and the obstacle avoidance capability by 50%.

[0010] Preferably, the adjacent field of view of the high-definition camera device overlaps by 10% to 20%, enabling the generation of a 360° top-down panoramic image through inverse perspective transformation and image fusion.

[0011] Preferably, the central controller has a wireless data transmission device for transmitting sensor data to a remote monitoring system in real time.

[0012] Preferably, the high-definition camera device, ultrasonic radar device, lidar device, and central controller are connected via a CAN bus.

[0013] Preferably, a high-precision encoder is provided at the joint connection between the boom steering wheel, the boom, the arm hydraulic motor steering wheel, the arm, and the bucket, and the high-precision encoder is communicatively connected to the central controller.

[0014] The present invention also provides a biomimetic training method for the above-mentioned equipment based on the fusion of manual operation and deep learning, comprising the following steps:

[0015] (1) During the manual demonstration stage, operation data is collected using a high-definition camera device, an ultrasonic radar device, a lidar device (24), and a high-precision encoder.

[0016] (2) Based on deep reinforcement learning, an intelligent mining decision model is constructed, the operation data collected in step (1) is normalized, and the temporal features are extracted through a long short-term memory network (LSTM) to model the dynamic environmental changes in the mining process.

[0017] (3) Based on the dynamic coordination control of human-machine collaboration, the model performance is continuously optimized. A shared control strategy is adopted to generate the final control command by combining the manual operation command and the model output through adaptive weights. (4) An extended Kalman filter is established to fuse multi-source sensor data, and a digital twin is constructed to realize virtual-real interaction and verify the system security. Finally, the optimized hybrid enhanced intelligent model is integrated into the equipment control system to realize the autonomous operation of complex and ever-changing foundation pit excavation.

[0018] Preferably, step (2) specifically includes:

[0019] Use the following formula to extract time series features:

[0020] h t =LSTM(x t ,h t-1 );

[0021] h t The output state at the current moment, x t For the current input, h t-1 This refers to the output state at the previous moment;

[0022] A priority experience replay buffer is constructed using human operation demonstration data, and the policy network is updated using importance sampling weighting:

[0023]

[0024] Where A(s,a) is the advantage function, η is the learning rate, and Δθ is the update amount of the policy network parameter θ in the current training step. To calculate the gradient of the policy network parameters θ, π θ (a|s) represents the probability distribution of the policy network selecting action a, where a is the specific operation performed in the environment, and s is the current state of the environment.

[0025] After the first round of manual operation, the distribution differences between the model prediction and the measured data are compared to gradually approach the optimal control strategy.

[0026] Preferably, step (3) specifically includes:

[0027] The following formula is used to control the generation of commands:

[0028] u Cmd =αu human +(1-α)u DRL

[0029] Where: u Cmd The system generates fusion control commands, α is a weighting coefficient with a value range of 0-1, and u... h uman For control commands directly input by manual operation, u DRL Autonomous control instructions generated by deep reinforcement learning;

[0030] The adaptive weights are dynamically coordinated, gradually transitioning from a higher proportion of manual operation to a higher proportion of machine operation. At the same time, the process is iteratively optimized. After each job cycle, the performance of the old and new models is compared, and the job complexity is gradually increased, so as to achieve a smooth transition from "human-led" to "autonomous decision-making".

[0031] Beneficial effects

[0032] As can be seen from the above technical solution, compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. Based on the traditional boom-boom-bucket three-joint design, this invention adds a boom rotation mechanism, which can greatly increase the working range of the robotic arm. Combined with multi-sensor fusion data, it can greatly improve the success rate of obstacle avoidance in complex, narrow and confined spaces, increasing the overall excavation area of ​​the equipment by 30% and the obstacle avoidance capability by 50%.

[0034] 2. This invention integrates multiple image acquisition devices such as ultrasonic radar, high-definition camera, and lidar, enabling the acquisition of three-dimensional data of the surrounding environment of the subway foundation pit excavation equipment from multiple angles and modes. This significantly improves the accuracy and completeness of spatial perception, thereby achieving intelligent obstacle avoidance and efficient path planning, ultimately improving excavation efficiency and reducing over-excavation. Furthermore, this invention proposes a deep reinforcement learning method based on human collaboration for unmanned operation of foundation pit excavation equipment, which helps to improve the level of unmanned operation in excavation.

[0035] 3. This invention employs ultrasonic radar, which maintains a high obstacle detection rate even in complex construction environments such as high dust concentration, low light, and rain / fog. It utilizes the high-frequency sound wave reflection characteristics to analyze obstacle distance and location information in real time, compensating for the blind spots of optical sensors in such scenarios and ensuring normal operation and safe construction under harsh conditions. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0037] Figure 1 This is a front view of a subway foundation pit excavation equipment based on multi-joint linkage according to an embodiment of the present invention;

[0038] Figure 2 This is a top view of a subway foundation pit excavation equipment based on multi-joint linkage according to an embodiment of the present invention.

[0039] Figure 3 This is a schematic diagram showing the installation positions of the ultrasonic radar device and the high-definition camera device according to an embodiment of the present invention;

[0040] Figure 4 This is an isometric schematic diagram of the transportation device according to an embodiment of the present invention.

[0041] In the diagram: 1. Frame; 2. Control cabin; 3. Boom steering wheel; 4. Boom; 5. Arm hydraulic motor steering wheel; 6. Arm; 7. Bucket; 8. Transport device; 9. Tracked walking device; 10. Second ultrasonic radar device; 13. Third ultrasonic radar device; 12. First ultrasonic radar device; 13. Fourth ultrasonic radar device; 14-18. High-definition camera device; 19. Transport trough; 20. Scraper; 21. Traction pin; 22. Sprocket; 23. Chain; 24. LiDAR; 25. Central controller. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example:

[0044] like Figures 1-4 As shown in the figure, this embodiment is a subway foundation pit excavation equipment based on multi-joint linkage.

[0045] A subway foundation pit excavation equipment based on multi-joint linkage includes a frame 1 rotatably mounted on a tracked walking device 9; an operation cabin 2 and a boom steering wheel 3 fixedly mounted at the front end of the frame 1; a boom 4 rotatably connected to the boom steering wheel 3; a forearm hydraulic motor steering wheel 5 rotatably connected to the boom 4; a forearm 6 rotatably connected to the forearm hydraulic motor steering wheel 5; a bucket 7 rotatably connected to the forearm 6; and a transport device 8 connected to the frame 1; the control cabin 2 is located at its front bottom, the bottom right side of the frame, and the outer left side of the frame. An ultrasonic radar device is installed on the outer surface of the right side; a laser radar device 24 is installed on the upper surface of the control cabin 2; high-definition camera devices are installed on the front, left, right, rear left and rear right sides of the frame 1; the boom has a rotation range of 60° to the left and right; the boom hydraulic motor steering wheel (5) enables the boom to have a rotation range of 90° to the left and right; high-precision encoders are installed at the joint connections between the boom steering wheel 3, boom 4, boom hydraulic motor steering wheel 5, boom 6 and bucket 7.

[0046] The ultrasonic radar device comprises four independent units: a first ultrasonic radar device 12, a second ultrasonic radar device 10, a third ultrasonic radar device 11, and a fourth ultrasonic radar device 13; please refer to [link to relevant documentation]. Figure 3 The first ultrasonic radar device 12 is fixedly installed at the bottom front side of the control cabin 2; the second ultrasonic radar device 10 is fixedly installed on the mounting plane at the bottom right side of the frame 1; the third ultrasonic radar device 11 is fixedly installed on the outer left side of the frame 1; and the fourth ultrasonic radar device 13 is fixedly installed on the outer right side of the frame 1.

[0047] Five independent high-definition camera devices 14-18 are installed on the front, left, right, rear left and rear right sides of the frame 1. Each high-definition camera device includes an adjustable mounting base and a high-definition camera, as is well known to those skilled in the art. The adjustable mounting base is detachably bolted to the frame 1, and the high-definition camera is rotatably mounted on the adjustable mounting base.

[0048] Figure 4 This is an isometric schematic diagram of the transport device in an embodiment. The transport device includes a transport trough 19, a scraper 20, a traction pin 21, a sprocket 22, and a chain 23, all connected to each other. The transport trough 71 includes a scraper 72, a traction pin 73, a sprocket 74, and a chain 75. The scraper 72 is hinged to the chain 75 via the traction pin 73. The chain 75 meshes with the circumferential outer edge of the sprocket 74. The sprocket 74 is driven by a motor, causing the scraper 72 to reciprocate along the extension direction of the transport trough 71, transporting the excavated soil from the bucket 6 to the rear of the equipment. The tilt angle of the transport device 7 can be controlled by a lifting cylinder.

[0049] The central controller 25 is connected to the ultrasonic radar device, the high-definition camera device, the lidar device, and the high-precision encoder via a CAN bus.

[0050] In this embodiment, the frame 1, control cabin 2, boom steering wheel 3, boom 4, boom hydraulic motor steering wheel 5, boom 6, bucket 7, transport device 8, tracked walking device 9, lidar 24, central controller 25, ultrasonic radar device, and high-definition camera device 14-18; the transport trough 19, scraper 20, traction pin 21, sprocket 22, chain 23, and high-precision encoder all adopt existing products or structures well known to those skilled in the art, and their interconnections also adopt existing connection methods or control methods well known to those skilled in the art.

[0051] Ultrasonic radar can detect obstacles in front of and to the sides of excavating equipment in dark, smoke, and dusty environments without damaging the object being measured. LiDAR can collect terrain data during earthwork excavation using multi-echo ranging technology, and generate high-precision 3D point clouds by combining GNSS / IMU positioning and attitude compensation. After denoising, registration, and digital elevation modeling, the changes in terrain before and after excavation are compared and analyzed to accurately calculate the earthwork volume. Path planning algorithms are also integrated to optimize the machinery's operating trajectory. For five independent high-definition camera devices, the field of view of adjacent high-definition cameras overlaps by 10%–20%, ensuring seamless coverage of blind spots. The five wide-angle images are mapped to a top-view plane through inverse perspective transformation, and multi-band fusion is used to eliminate seams, generating a seamless 360° top-view panoramic image. In the stitched overlapping areas, pixel weights are dynamically adjusted according to the distance between the camera and the target, with closer cameras receiving higher weights. Combined with obstacle detection results from the ultrasonic radar device, collision risks are eliminated. Based on ultrasonic radar, lidar, and high-definition camera devices, real-time environmental information and the location information of the mining equipment are accurately obtained.

[0052] The image data captured by the camera is processed, and multiple images with overlapping areas are combined into one image using image stitching. First, the scale space extrema are found using the Difference of Gaussian Pyramid (DoG).

[0053] D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]·I(x,y)

[0054] Where G is the Gaussian kernel, I is the image, and σ is the scale.

[0055] Then feature matching and transformation estimation are performed. Feature matching uses the nearest neighbor distance ratio, i.e., NNDR.

[0056]

[0057] Where d1 and d2 are the nearest and second nearest Euclidean distances, respectively, and T is the threshold (usually 0.6-0.8). Finally, image fusion is performed using a weighted average fusion: in the overlapping region, the pixel value is calculated by weighting the values ​​from both images.

[0058] I blend (x,y)=αI1(x,y)+ ( 1-α)I2(x,y)

[0059] Where the weight α varies with the distance from the pixel to the image boundary, I blend (x,y) is the final intensity value of the fused image at pixel coordinates (x,y), α is the weighting coefficient, and I1(x,y) and I2(x,y) are the pixel intensity values ​​of the first and second images at coordinates (x,y), respectively.

[0060] The point cloud data acquired by the lidar is processed as follows: First, the original point cloud is preprocessed, mainly including downsampling and noise reduction. Then, core processing is performed, including segmentation, feature extraction, registration and classification. Through the above process, the original disordered and noisy point cloud data can be transformed into information-rich structured data.

[0061] See Figure 1 Based on the traditional boom-arm-bucket three-joint design, and building upon the original 60° boom rotation angle, a hydraulic motor steering wheel 5 for the arm has been added, enabling the arm to rotate a total of 90° to the left and right. The boom, arm, hydraulic motor steering wheel 5, and arm form a tandem robotic arm, further upgrading the motion model.

[0062] T=T1(θ1)T2(θ2)T3(θ3)T4(θ4)

[0063] Wherein, T4(θ4) is the transformation matrix of the boom slewing mechanism, T1(θ1) is the transformation matrix of the bucket slewing mechanism, T2(θ2) is the transformation matrix of the boom and arm slewing mechanisms, and T3(θ3) is the transformation matrix of the arm slewing mechanism. This significantly improves the reachable working space and obstacle avoidance capability of the end effector, increasing the overall excavation area by 30% and obstacle avoidance capability by 50%. The equipment in this embodiment is suitable for excavating foundation pits in narrow and harsh environments, but is not limited to such pits; it is also applicable to multi-pillar environments. Furthermore, when operating in large foundation pit environments, the 30% increase in excavation area significantly improves excavation efficiency and greatly shortens the construction period. Simultaneously, by combining real-time information about the surrounding environment, collision-free path planning can be achieved. Combined with human-machine collaborative dynamic coordination micro-operation training and online learning mechanisms, autonomous excavation of foundation pits under complex working conditions can ultimately be realized.

[0064] This embodiment also provides a method for training the above-mentioned biomimetic device model, which specifically includes the following steps:

[0065] (1) Manually operate equipment to obtain real-time operation data

[0066] First, the manual operation device performs actual excavation work, while the high-precision encoder installed at the joints obtains the real-time joint angle and calculates the end-effector pose of the robotic arm using a model familiar to those skilled in the art; the surrounding environment, including pillars, soil piles, and personnel, is determined by the high-precision encoder, lidar device, high-definition camera device, and ultrasonic radar device.

[0067] (2) Constructing an intelligent mining decision model based on deep reinforcement learning

[0068] The signals acquired by the high-precision encoder, lidar device, high-definition camera device, and ultrasonic radar device are normalized, and temporal features are extracted using a Long Short-Term Memory (LSTM) network, which is well known to those skilled in the art, to model and mine the dynamic environmental changes during the process.

[0069]

[0070] Where: h t The output state at the current moment, x t For the current input, h t-1 This represents the output state at the previous moment.

[0071] A priority experience replay buffer is constructed using human operation demonstration data, and the policy network is updated using importance sampling weighting:

[0072]

[0073] Where A(s,a) is the advantage function, η is the learning rate, and Δθ is the update of the policy network parameter θ in the current training step. To calculate the gradient of the policy network parameters θ and π θ (a|s) represents the probability distribution of the policy network selecting action a, where a is the specific operation performed in the environment, and s is the current state of the environment.

[0074] After the first round of manual operation, the distribution differences between the model prediction and the measured data are compared to gradually approach the optimal control strategy.

[0075] (3) Continuous optimization of model performance based on human-machine collaborative dynamic coordination control

[0076] A shared control strategy is adopted, which uses adaptive weights to generate the final control command by combining manual operation instructions with model output:

[0077] u Cmd =αu h uman +(1-α)u DRL

[0078] The adaptive weights are dynamically coordinated, gradually transitioning from a higher proportion of manual operation to a higher proportion of machine operation. At the same time, the process is iteratively optimized. After each job cycle, the performance of the old and new models is compared, and the job complexity is gradually increased, so as to achieve a smooth transition from "human-led" to "autonomous decision-making".

[0079] (4) Implement adaptive excavation operations under miscellaneous working conditions

[0080] An extended Kalman filter is used to fuse multi-source sensor data, a digital twin is constructed to achieve virtual-real interaction, and the system security is verified. Finally, the optimized hybrid enhanced intelligent model is integrated into the equipment control system to achieve autonomous operation of complex and ever-changing foundation pit excavation.

[0081] In this specification, the apparatus disclosed in the embodiments corresponds to the methods disclosed in the embodiments, and the relevant parts can be referred to in the method section.

[0082] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A subway foundation pit excavation equipment based on multi-joint linkage, characterized in that, include: A frame (1) that is rotatably mounted on a tracked walking device (9); The control cabin (2) and boom steering wheel (3) are fixedly installed at the front end of the frame (1); the boom (4) is rotatably connected to the boom steering wheel (3); the boom hydraulic motor steering wheel (5) is rotatably connected to the boom (4); the boom (6) is rotatably connected to the boom hydraulic motor steering wheel (5); the bucket (7) is rotatably connected to the boom (6); and the transport device (8) is connected to the frame (1). An ultrasonic radar device is installed on the bottom front side of the control cabin (2), the bottom right side of the frame, the outer surface of the left side of the frame, and the outer surface of the right side of the frame; a laser radar device (24) is installed on the upper surface of the control cabin (2); a high-definition camera device is installed on the front, left, right, rear left and rear right sides of the frame (1); the boom has a rotation range of 60° to the left and right. The forearm hydraulic motor steering wheel (5) enables the forearm to have a rotation range of 90° to the left and right; the high-definition camera device, ultrasonic radar device, and lidar device (24) are communicatively connected to the central controller (25) in the control cabin (2); the upper arm (4), the forearm hydraulic motor steering wheel (5), and the forearm (6) form a series robotic arm, upgrading the motion model to: T=T1(θ1)T2(θ2)T3(θ3)T4(θ4) Wherein: T4(θ4) is the transformation matrix of the boom slewing mechanism, T1(θ1) is the transformation matrix of the bucket slewing mechanism, T2(θ2) is the transformation matrix of the boom and arm slewing mechanisms, and T3(θ3) is the transformation matrix of the arm slewing mechanism; these are used to improve the reachable working space and obstacle avoidance capability of the end effector, thereby increasing the overall excavation area of ​​the equipment by 30% and the obstacle avoidance capability by 50%.

2. The subway foundation pit excavation equipment based on multi-joint linkage according to claim 1, characterized in that, The adjacent field of view of the high-definition camera device overlaps by 10% to 20%, and can generate a 360° top-down panoramic image through inverse perspective transformation and image fusion.

3. The subway foundation pit excavation equipment based on multi-joint linkage according to claim 1, characterized in that, The central controller (25) has a wireless data transmission device for transmitting sensor data to a remote monitoring system in real time.

4. The subway foundation pit excavation equipment based on multi-joint linkage according to claim 1, characterized in that, The high-definition camera device, ultrasonic radar device, and lidar device (24) are connected to the central controller (25) via a CAN bus.

5. The subway foundation pit excavation equipment based on multi-joint linkage according to claim 1, characterized in that, A high-precision encoder is provided at the joint connection between the boom steering wheel (3), boom (4), boom hydraulic motor steering wheel (5), boom (6), and bucket (7), and the high-precision encoder is connected to the central controller (25).

6. A biomimetic training method based on the fusion of manual operation and deep learning, used in the subway foundation pit excavation equipment based on multi-joint linkage as described in any one of claims 1-5, characterized in that, Includes the following steps: (1) During the manual demonstration stage, operation data is collected using a high-definition camera device, an ultrasonic radar device, a lidar device (24), and a high-precision encoder. (2) Based on deep reinforcement learning, an intelligent mining decision model is constructed, the operation data collected in step (1) is normalized, and the temporal features are extracted through a long short-term memory network (LSTM) to model the dynamic environmental changes in the mining process. (3) Based on human-machine collaborative dynamic coordination control to continuously optimize model performance, a shared control strategy is adopted to generate the final control command by combining manual operation instructions and model output through adaptive weights. (4) Establish extended Kalman filter to fuse multi-source sensor data, construct a digital twin to realize virtual-real interaction, and verify system security; finally, integrate the optimized hybrid enhanced intelligent model into the equipment control system to realize autonomous operation of complex and ever-changing foundation pit excavation.

7. The biomimetic training method based on the fusion of manual operation and deep learning according to claim 6, characterized in that, Step (2) specifically includes: Use the following formula to extract time series features: h t =LSTM(x t ,h t-1 ); Where: h t The output state at the current moment, x t For the current input, h t-1 This refers to the output state at the previous moment; A priority experience replay buffer is constructed using human operation demonstration data, and the policy network is updated using importance sampling weighting: Where: A(s,a) is the advantage function, η is the learning rate, and Δθ is the update amount of the policy network parameter θ in the current training step. To calculate the gradient of the policy network parameters θ and π θ (a|s) represents the probability distribution of the policy network selecting action a, where a is the specific operation performed in the environment, and s is the current state of the environment. After the first round of manual operation, the distribution differences between the model prediction and the measured data are compared to gradually approach the optimal control strategy.

8. The biomimetic training method based on the fusion of manual operation and deep learning according to claim 6, characterized in that, Step (3) specifically includes: The following formula is used to control the generation of commands: you Cmd =au human +(1-a)u DRL Where: u Cmd The system generates fusion control commands, α is a weighting coefficient with a value range of 0-1, and u... human For control commands directly input by manual operation, u DRL Autonomous control instructions generated by deep reinforcement learning; The adaptive weights are dynamically coordinated, gradually transitioning from a higher proportion of manual operation to a higher proportion of machine operation. At the same time, the process is iteratively optimized. After each job cycle, the performance of the old and new models is compared, and the job complexity is gradually increased, so as to achieve a smooth transition from "human-led" to "autonomous decision-making".

Images