Multi-modal perception and reinforcement learning engineering machinery intelligent regulation and control method and system

Abstract

The invention discloses a multi-mode perception and reinforcement learning engineering machinery intelligent regulation and control method and system, and belongs to the technical field of intelligent control and industrial automation. According to the system, stratum data within the range of 20-50 meters in front of a shield tunneling machine are collected in real time through multi-mode sensing equipment such as a distributed optical fiber sensor, a cutterhead vibration sensor and an electromagnetic wave radar, and original data are processed by adopting a wavelet-Fourier combined noise reduction algorithm; and inputting the processed multi-modal data into a reinforcement learning intelligent decision-making model based on a CNN-LSTM hybrid network architecture. The model is trained through a dynamic reward function, and weight coefficient combinations can be automatically switched according to different construction scenes. According to the system, parameters such as the rotating speed, the thrust and the grouting amount of a shield cutter head are regulated and controlled in real time through the self-adaptive control module, and the slurry utilization rate is increased through the gradient pulse grouting technology. The precision, efficiency and safety of shield construction are remarkably improved, and meanwhile energy consumption and construction cost are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control and industrial automation, and particularly relates to an intelligent regulation method and system for construction machinery based on multi-modal perception and reinforcement learning. Background Art

[0002] With the rapid development of industrial automation and intelligent control technologies, the intelligent regulation of construction machinery has become an important research direction in the modern construction field. The control methods of traditional construction machinery mainly rely on preset rules and single-sensor feedback, and it is difficult to cope with complex and changeable construction environments. For example, in underground tunnel construction, the geological conditions are complex and changeable, and the traditional control system cannot perceive the changes in the front stratum in real time, resulting in problems such as low construction efficiency, high energy consumption, and poor safety. In addition, existing control methods mostly adopt simple control strategies such as the PID algorithm, lacking the comprehensive optimization ability for multiple objectives (such as energy consumption, efficiency, and safety), and it is difficult to meet the high-efficiency, intelligent, and safe requirements of modern construction.

[0003] In recent years, with the progress of big data, artificial intelligence, and Internet of Things technologies, data-driven intelligent control systems have gradually become a research hotspot. In the prior art, some studies have tried to apply machine learning algorithms to the control optimization of construction machinery, but these methods mostly rely on a single data source and cannot achieve the fusion and real-time decision-making of multi-modal data. For example, the commonly used vibration sensors or fiber optic sensors in the prior art can only provide local information and lack a comprehensive perception of the overall state of the stratum. In addition, existing multi-objective optimization methods usually only adjust a single parameter and lack the dynamic balance of energy consumption, efficiency, and safety, resulting in poor performance of the system in practical applications.

[0004] In the field of control and regulation systems, the control systems in the prior art mostly adopt fixed parameters or simple feedback regulation mechanisms, and it is difficult to adapt to the dynamic changes in complex environments. For example, although the traditional PID control can achieve real-time adjustment of local parameters, its adjustment accuracy and stability significantly decrease when facing multi-variable and non-linear systems. In addition, the scalability of existing control systems is poor, and it is difficult to perform data interaction with external platforms (such as BIM platforms), which limits the intelligent level and application scope of the system. Therefore, there is an urgent need for an intelligent regulation system that can fuse multi-modal perception data, achieve multi-objective dynamic optimization, and have good scalability to improve the construction efficiency and safety of construction machinery. Summary of the Invention

[0005] To solve the above technical problems, an intelligent regulation method for construction machinery based on multi-modal perception and reinforcement learning is proposed, including obtaining multi-modal perception data within the operation area of the shield machine;

[0006] Input the multi-modal perception data into a preset reinforcement learning intelligent decision-making model;

[0007] According to the output of the reinforcement learning intelligent decision-making model, the working parameters such as the rotation speed, thrust and grouting volume of the shield cutter head are adjusted in real time through an adaptive control module.

[0008] As a preferred scheme of the intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to the present invention, wherein: the multi-modal perception data includes: distributed optical fiber sensing data, cutter head vibration monitoring data, electromagnetic wave radar detection data and temperature monitoring data; the multi-modal perception data is the formation strain, temperature, moisture content and fracture distribution data within a range of 20-50 meters in front of the shield machine collected in real time by a multi-modal perception module.

[0009] As a preferred scheme of the intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to the present invention, wherein: the reinforcement learning intelligent decision-making model adopts a CNN-LSTM hybrid network architecture, which is used to predict the formation strength grade and risk and generate an optimized decision for the cutter head working parameters.

[0010] As a preferred scheme of the intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to the present invention, wherein: the adaptive control module adjusts the working parameters of the shield cutter head in real time, including: adjusting the addition ratio of the accelerating agent through a PID closed-loop control system; monitoring the rotation speed, thrust and grouting pressure of the shield cutter head in real time and making dynamic adjustments according to the actual construction environment; the double-fluid grouting system is equipped with a conductivity sensor and a viscosity sensor to feedback the slurry mixing state in real time.

[0011] As a preferred scheme of the intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to the present invention, wherein: the reinforcement learning intelligent decision-making model is trained based on a dynamic reward function, and the weight coefficients of the dynamic reward function are automatically switched to different preset parameter combinations according to the construction scenario: the first parameter combination is adopted in the normal tunneling stage; the second parameter combination is adopted when passing through sensitive areas; the third parameter combination is adopted in the water-rich sand layer stage.

[0012] As a preferred scheme of the intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to the present invention, wherein: the method further includes a system expansion step: performing data exchange and cooperation with an external system through an open API interface; loading a user-defined control strategy through an SDK toolkit to correct the construction parameters of the shield cutter head in real time; the open API interface supports the RESTful protocol and the MQTT protocol and is compatible with the data access of the BIM platform, geological radar and sonar equipment.

[0013] As a preferred solution of the intelligent control method for construction machinery based on multi-modal perception and reinforcement learning according to the present invention, wherein: the SDK toolkit is built with a visual policy editor, allowing users to define the following control modes by dragging and dropping components: settlement-sensitive mode: the settlement threshold setting range is ±1.0 - 2.0 mm; energy-saving priority mode: the energy consumption weight is higher than other weights; hard rock tunneling mode: the thrust upper limit ≥ 20 Mpa, and the cutter head rotation speed ≤ 1.2 rpm.

[0014] Another object of the present invention is to provide an intelligent control system for construction machinery based on multi-modal perception and reinforcement learning. The present invention solves the problem that although the traditional PID control can achieve real-time adjustment of local parameters, when facing multi-variable and non-linear systems, its adjustment accuracy and stability are significantly reduced. In addition, the scalability of the existing control system is poor, and it is difficult to perform data interaction with external platforms (such as BIM platforms), which limits the intelligent level and application range of the system.

[0015] As a preferred solution of the intelligent control system for construction machinery based on multi-modal perception and reinforcement learning according to the present invention, it is characterized in that it includes a data acquisition module for acquiring multi-modal perception data within the operation area of the shield machine;

[0016] A processing module for inputting the multi-modal perception data into a preset reinforcement learning intelligent decision-making model;

[0017] A control module for, according to the output of the reinforcement learning intelligent decision-making model, performing real-time control on working parameters such as the rotation speed, thrust, and grouting volume of the shield cutter head through an adaptive control module.

[0018] A computer device, including a memory and a processor, the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the intelligent control method for construction machinery based on multi-modal perception and reinforcement learning are implemented.

[0019] A computer-readable storage medium, on which a computer program is stored, and is characterized in that when the computer program is executed by a processor, the steps of the intelligent control method for construction machinery based on multi-modal perception and reinforcement learning are implemented.

[0020] The beneficial effects of the present invention: Through the organic combination of multi-modal perception, deep learning models, adaptive control, and expansion interfaces, the present invention constructs a comprehensive and intelligent construction machinery control system. This system realizes the full-closed-loop intelligent control from perception to decision-making and then to control, overcomes many limitations of the traditional shield machine control system when facing complex geological environments, significantly improves construction accuracy, efficiency, and safety, while reducing energy consumption and construction costs.

[0021] In particular, through the combination of reinforcement learning and multi-modal perception, the present invention achieves precise perception and intelligent response to complex and changeable geological environments, enabling the shield machine to "foresee" and adapt to geological changes ahead in advance, rather than the passive response in the traditional sense. This predictive control ability is an unexpected technical effect brought by the present invention, which fundamentally changes the working mode of the shield machine, enabling it to maintain the best working state in various complex environments and significantly improving the overall level of underground engineering construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0023] Figure 1 It is the overall flowchart of an intelligent control method for construction machinery with multi-modal perception and reinforcement learning provided by an embodiment of the present invention.

[0024] Figure 2 It is the computer equipment diagram of an intelligent control method for construction machinery with multi-modal perception and reinforcement learning provided by an embodiment of the present invention.

[0025] Figure 3 It is the schematic diagram of the application scenario of an intelligent control method for construction machinery with multi-modal perception and reinforcement learning provided by an embodiment of the present invention.

[0026] Figure 4 It is the overall block diagram of an intelligent control system for construction machinery with multi-modal perception and reinforcement learning provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will provide a detailed description of the specific embodiments of the present invention in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0028] Example 1, referring to Figure 1 , which is the first embodiment of the present invention. This embodiment provides an intelligent control method for construction machinery with multi-modal perception and reinforcement learning, including:

[0029] S100: Obtain multi-modal perception data within the operation area of the shield machine;

[0030] In an embodiment of the present invention, the operation area of the shield machine refers to the underground area within a range of 20 - 50 meters in front of the advancing direction of the shield machine. The geological conditions in this area are crucial for the tunneling work of the shield machine. Timely and accurately obtaining the multi-modal perception data in this area can effectively improve the operation efficiency and safety of the shield machine.

[0031] Specifically, the acquisition process of the multi-modal perception data is completed by multiple perception devices installed at the head of the shield machine. These perception devices can be divided into four categories: distributed optical fiber sensor system, cutter head vibration monitoring system, electromagnetic wave radar detection system, and temperature monitoring system. Each perception device collects data for different physical parameters, jointly constituting a comprehensive formation information perception network.

[0032] In a preferred embodiment, the distributed optical fiber sensor system adopts Brillouin Optical Time Domain Reflectometry (BOTDR) technology. By arranging optical fibers along the tunnel axis around the shield machine, it can real-time monitor the strain and temperature changes along the fiber distribution. The spatial resolution of this system can reach 0.5 meters, the strain measurement accuracy is ±10με, the temperature measurement accuracy is ±0.5°C, and the sampling frequency is 10Hz. This high-precision optical fiber sensing network can capture subtle formation deformations, providing accurate basic data for subsequent analysis.

[0033] The cutter head vibration monitoring system is realized by installing triaxial acceleration sensors and displacement sensors at key parts of the cutter head. The sampling frequency of each sensor is set to 1000Hz, which can capture vibration signals in the frequency range of 0 - 500Hz. A total of 12 measurement points are arranged on the cutter head at different angular and radial positions, forming an all-round vibration monitoring network. These vibration data not only reflect the contact state between the cutter head and the formation, but also can indirectly reflect the hardness and uniformity of the formation.

[0034] The electromagnetic wave radar detection system adopts Frequency Modulated Continuous Wave (FMCW) technology, with a working frequency range of 1 - 3GHz and a maximum detection depth of up to 50 meters. By emitting electromagnetic waves and receiving the reflected waves from the formation, analyzing the time delay and amplitude change of the reflected waves, it can detect anomalies, cavities, water-bearing areas, and rock layer interfaces in the front formation, etc. The spatial resolution of the system can reach 0.1 meters, enabling precise positioning of potential geological risks.

[0035] The temperature monitoring system includes a temperature sensor network distributed at different parts of the shield machine, monitoring the surface temperature of the cutter head, the temperature of the main bearing, the temperature of the hydraulic oil, and the temperature of the surrounding soil, etc. On the one hand, these temperature data can monitor the operation state of the shield machine itself, and on the other hand, by analyzing the temperature gradient change, it can infer the formation moisture content and thermal physical parameters.

[0036] During the data acquisition process, all sensing systems adopt a synchronous triggering mechanism to ensure the consistency of data from different modalities in the time dimension. The data acquisition controller is set with two working modes: normal mode and high-frequency mode. During the normal tunneling stage, the system acquires data at a frequency of 1 Hz; while in the case of encountering complex strata or abnormal conditions, it will automatically switch to the high-frequency mode, and the acquisition frequency is increased to 10 Hz to capture more transient change characteristics.

[0037] It should be noted that the acquisition of multi-modal sensing data is the basic link of the present invention. Only by obtaining comprehensive and accurate sensing data can a reliable basis be provided for subsequent intelligent decision-making and control. By combining sensing technologies based on different physical principles, the system can sense the stratum state from multiple dimensions, overcoming the limitations of a single sensing method and greatly improving the comprehensiveness and reliability of stratum information acquisition.

[0038] S101: The multi-modal sensing data includes: distributed fiber optic sensing data, cutterhead vibration monitoring data, electromagnetic wave radar detection data, and temperature monitoring data; the multi-modal sensing data is the stratum strain, temperature, moisture content, and fracture distribution data within a range of 20 - 50 meters in front of the shield machine collected in real time by the multi-modal sensing module.

[0039] In the embodiments of the present invention, the types and contents of multi-modal sensing data are diverse, and each type of data carries specific stratum information. The following provides a detailed description of various types of data.

[0040] The distributed fiber optic sensing data mainly includes strain and temperature data distributed along the optical fiber. The strain data is in microstrain (με) units and reflects the deformation degree of the stratum; the temperature data is in degrees Celsius (°C) and reflects the temperature distribution of the stratum. The strain data is usually stored in the form of a 1000×N matrix, where N is the number of measurement points of the optical fiber. Each row of the matrix represents the sampled data at a time point, and each column represents the measured value at a spatial position. By analyzing the spatio-temporal distribution characteristics of the strain data, the stress state, deformation trend, and abnormal areas of the stratum can be inferred.

[0041] The cutterhead vibration monitoring data includes the acceleration and displacement time series of each measurement point on the cutterhead. The acceleration data is in g (acceleration due to gravity) units, and the displacement data is in millimeters (mm). These data are usually stored in the form of time series, and each measurement point corresponds to three-direction (X, Y, Z) data series. By analyzing the spectral characteristics and time-domain characteristics of the vibration data, the contact state between the cutterhead and the stratum, the hardness distribution of the stratum, and potential abnormal conditions (such as encountering boulders) can be judged.

[0042] The electromagnetic wave radar detection data is stored in the form of radar images, showing the variation relationship of the reflected wave amplitude with time (or depth). These data are usually organized in a two-dimensional matrix, where the rows of the matrix represent the detection angles or positions, the columns represent the depth, and the values of the matrix elements represent the amplitude or phase of the reflected wave. By analyzing the reflection patterns in the radar images, interfaces, anomalies, voids, and water-bearing areas in the front stratum can be identified.

[0043] The temperature monitoring data includes the temperature values of various parts of the shield machine and the surrounding soil mass, in degrees Celsius (°C). These data are usually stored in the form of a time series, and each measurement point corresponds to a temperature series. By analyzing the spatial distribution and temporal variation of the temperature, the operating state of the shield machine can be monitored, and at the same time, the thermal physical properties and water content of the stratum can be inferred.

[0044] In terms of data storage and transmission, a hierarchical design strategy is adopted. The original perception data is first stored in the local data cache module for a short time, and then after preprocessing, it is transmitted to the local server for further analysis and decision-making. To ensure the real-time and reliable data transmission, the system adopts a dual-channel redundancy design, including a wired channel (industrial Ethernet) and a wireless channel (5G network). The AES-256 encryption algorithm is used to ensure data security during the data transmission process, and the transmission protocol adopts a custom protocol based on MQTT, supporting the priority transmission and resume of interrupted transmission of data.

[0045] In a preferred embodiment, the system also introduces a data quality assessment mechanism to perform real-time assessment on the acquired multi-modal perception data. The assessment indicators include data integrity, consistency, signal-to-noise ratio, and timeliness, etc. When it is detected that the data quality of a certain sensor deteriorates, the system will automatically adjust the weight of the sensor or enable a backup sensor to ensure the stability of the overall data quality.

[0046] It should be noted that the multi-modal perception data is the basis for the intelligent decision-making of the present invention. Different types of data reflect the stratum state from different angles, complementing and verifying each other, and can provide more comprehensive and reliable stratum information. Through multi-modal data fusion, the system can overcome the limitations of a single data source, improve the accuracy and robustness of perception, and provide a solid data foundation for subsequent intelligent decision-making.

[0047] S200: Input the multi-modal perception data into a preset reinforcement learning intelligent decision model;

[0048] In the embodiment of the present invention, inputting the multi-modal perception data into the reinforcement learning intelligent decision model is the core link to achieve intelligent regulation. This process involves multiple steps such as data preprocessing, feature extraction, and model inference. These steps will be described in detail below.

[0049] First, the multi-modal perception data needs to be preprocessed before being input into the model, including operations such as data cleaning, normalization, and alignment. Data cleaning aims to remove outliers and noise, and the methods used include median filtering in a sliding window, wavelet transform for noise reduction, etc. For distributed fiber optic sensing data, a Butterworth band-pass filter is used to filter the original signal, with a filtering frequency band of 0.1 - 50 Hz, which can effectively remove high-frequency noise and low-frequency drift. For cutterhead vibration data, the wavelet packet decomposition method is used to extract the effective components in the frequency band of 0 - 500 Hz, while filtering out the noise components with the same frequency as the shield vibration (mainly concentrated in the frequency band of 50 - 150 Hz). For electromagnetic wave radar data, an adaptive depth gain compensation algorithm is used to correct the signal attenuation in the depth direction and improve the quality of deep reflection signals.

[0050] Normalization processing aims to eliminate the dimensional differences of different physical quantities, making various types of data comparable and fusible. For different types of perception data, different normalization strategies are adopted: for numerical data such as strain and temperature, the Z-score standardization method is used, that is, subtracting the mean and dividing by the standard deviation; for two-dimensional data such as radar images, the block histogram equalization method is used to enhance the local contrast. After normalization processing, the value ranges of various types of data are mapped to similar ranges, which is beneficial to subsequent feature extraction and model learning.

[0051] Data alignment aims to solve the problem of inconsistency of different sensor data in the time and space dimensions. Due to the differences in the sampling frequencies and coverage ranges of different sensors, it is necessary to perform spatio-temporal alignment processing so that various types of data can be analyzed under the same spatio-temporal reference system. In the time dimension, the linear interpolation method is used to unify the data with different sampling frequencies to a standard frequency of 10 Hz; in the space dimension, the geographic information system (GIS) technology is used to map the data under different spatial reference systems into a unified three-dimensional coordinate system.

[0052] The multi-modal data after preprocessing then needs to be subjected to feature extraction to convert the original data into feature vectors describing the formation characteristics. For distributed fiber optic sensing data, the extracted features include the strain gradient matrix, the maximum strain position, the strain change rate, etc.; for cutterhead vibration data, the extracted features include the main frequency of the vibration spectrum, the harmonic energy ratio, the change trend of the vibration intensity over time, etc.; for electromagnetic wave radar data, the extracted features include the reflection waveform features, the interface position features, the abnormal body recognition features, etc.; for temperature monitoring data, the extracted features include the temperature gradient, the temperature change rate, the temperature abnormal area, etc.

[0053] In a preferred embodiment, deep learning algorithms are used for feature extraction, specifically an autoencoder network. An autoencoder consists of two parts: an encoder and a decoder. The encoder maps high-dimensional input data to a low-dimensional feature space, and the decoder attempts to reconstruct the original input from the low-dimensional features. By minimizing the reconstruction error, the autoencoder can learn the essential feature representation of the data. For each type of modal data, a dedicated autoencoder structure is designed: for one-dimensional time series data (such as strain and vibration sequences), a one-dimensional convolutional autoencoder is used; for two-dimensional image data (such as radar images), a two-dimensional convolutional autoencoder is used. These autoencoders are pre-trained on a large amount of historical data and can effectively extract the latent features of various types of data.

[0054] The multi-modal data after feature extraction needs to be feature fused to integrate the features of different modalities into a unified representation. The present invention adopts a multi-modal feature fusion method based on an attention mechanism. Specifically, the system first calculates the importance weights of the features of each modality, and then performs weighted summation on the features of each modality according to these weights. The importance weights are calculated by an attention network, which can dynamically adjust the importance of different modal features according to the current construction state and historical experience to achieve adaptive fusion. For example, in hard rock areas, the system will assign a higher weight to vibration features; while in water-rich areas, the weights of temperature and radar features will be increased.

[0055] Finally, the fused features are input into a reinforcement learning intelligent decision-making model for inference. The inputs of the model include the multi-modal feature vector at the current moment and the feature sequence within the historical time window, and the output is the optimized decision of the cutterhead working parameters, such as cutterhead rotation speed, thrust, and grouting volume.

[0056] It should be noted that the preprocessing, feature extraction, and fusion processes of multi-modal perception data are crucial for the performance of the model. Through reasonable data processing and feature engineering, the system can extract valuable information from the messy raw data, provide high-quality inputs for the reinforcement learning model, and thus improve the accuracy and reliability of decision-making.

[0057] S201: The reinforcement learning intelligent decision-making model adopts a CNN-LSTM hybrid network architecture for predicting the formation strength grade and risk and generating optimized decisions on cutterhead working parameters.

[0058] In the embodiment of the present invention, the reinforcement learning intelligent decision-making model is the core component of the system, responsible for converting multi-modal perception data into optimized decisions on cutterhead working parameters. This model adopts an advanced CNN-LSTM hybrid network architecture, which can effectively process spatio-temporal data and make intelligent decisions. The architecture, training method, and decision-making mechanism of this model will be described in detail below.

[0059] The CNN-LSTM hybrid network architecture of the present invention consists of four main parts: a feature extraction network, a sequence processing network, a policy network, and a value network. The feature extraction network is mainly composed of a convolutional neural network (CNN) and is responsible for extracting spatial features from multimodal data; the sequence processing network is mainly composed of a long short-term memory network (LSTM) and is responsible for capturing the dynamic features of time series; the policy network and the value network are responsible for generating action policies and evaluating state values, respectively.

[0060] Specifically, the feature extraction network contains 4 convolutional blocks, and each convolutional block consists of two convolutional layers and one max-pooling layer. The convolutional layer uses a 3×3 convolutional kernel, and the activation function adopts ReLU. The input of the first convolutional block is a multimodal feature matrix, including a fiber optic strain gradient matrix, a vibration spectrum feature map, a radar reflection feature map, etc. The output channel numbers of the four convolutional blocks are 32, 64, 128, and 256 respectively, extracting higher-level feature representations layer by layer. The output of the last convolutional block undergoes global average pooling to generate a feature vector with a fixed dimension as the spatial feature representation.

[0061] The sequence processing network consists of two stacked LSTM units, and the hidden state dimension of each layer of LSTM is 128. The input of the LSTM network is the feature sequence within a time window (usually the past 10 time steps), and the output is the temporal encoding of the current state. LSTM can capture the change patterns of features over time, such as the strain growth trend, vibration mode changes, etc. These temporal features are crucial for predicting future formation conditions.

[0062] The outputs of the feature extraction network and the sequence processing network are concatenated into a unified state representation vector and then input into the policy network and the value network. The policy network consists of three fully connected layers, with hidden layer dimensions of 256 and 128 respectively, and the output layer dimension is equal to the dimension of the action space. The output of the policy network undergoes a Softmax activation function to generate an action probability distribution, representing the probabilities of selecting each action in the current state. The value network also consists of three fully connected layers, with the same hidden layer dimensions as the policy network, but the output layer has only one neuron, representing the estimation of the current state value.

[0063] In a preferred embodiment, the model introduces an attention mechanism to enhance the ability to perceive key features. Specifically, a spatial attention module is added during the CNN feature extraction process to help the model focus on important spatial regions; a temporal attention module is added during the LSTM sequence processing to help the model focus on key time points.

[0064] The training of the reinforcement learning model adopts the policy gradient method and specifically implements the Actor-Critic architecture. Among them, the policy network acts as the Actor and is responsible for selecting actions based on the current state; the value network acts as the Critic and is responsible for evaluating the state value to provide a benchmark for policy update. During the training process, the model interacts with the environment (the actual shield tunneling process or the simulation environment) and continuously adjusts the policy to maximize the cumulative reward. The reward function comprehensively considers three aspects: energy consumption, efficiency, and safety, and its form is:

[0065] R = αE Saving + βT advance - γS Ssettlement

[0066] Among them, E Saving is the energy-saving rate, T advance is the tunneling efficiency (m / h), S Ssettlement is the excessive settlement value (mm), and α, β, γ are weight coefficients, which are dynamically adjusted according to the construction scenario.

[0067] To accelerate the learning process of the model and improve its generalization ability, the present invention adopts a transfer learning strategy. Specifically, first, the feature extraction part of the model is pre-trained on a large amount of historical data, and then the entire model is fine-tuned on the actual data of the current project. This method can utilize historical experience, quickly adapt to the new construction environment, and significantly improve the learning efficiency and generalization ability of the model.

[0068] At the same time, to handle the uncertainties and complexities in actual construction, the model adopts an ensemble learning method. The system parallelly trains multiple models with different initializations and hyperparameters to form a model ensemble. When making a decision, the system will comprehensively consider the prediction results of multiple models and obtain the final decision through weighted voting or Bayesian methods. This ensemble strategy can reduce the bias of a single model and improve the stability and reliability of the decision.

[0069] During the application process of the model, the system implements an online learning mechanism and can continuously learn and optimize from real-time construction data. After each section of construction is completed, the system will evaluate the gap between the actual effect and the expectation and update the model parameters according to the evaluation results. This closed-loop optimization mechanism enables the model to continuously adapt to the changing formation conditions and construction requirements and maintain long-term effectiveness.

[0070] S300: According to the output of the reinforcement learning intelligent decision-making model, the working parameters such as the rotation speed, thrust, and grouting volume of the shield cutter head are real-time regulated through the adaptive control module.

[0071] In an embodiment of the present invention, the adaptive control module is a key link connecting intelligent decision-making and actual execution. It is responsible for converting the decisions of the reinforcement learning model into specific control instructions and adjusting the working parameters of the shield cutter head in real time. This module adopts an adaptive control strategy, which can flexibly adjust control parameters according to construction status and environmental changes to ensure the stability and efficiency of the construction process. The structure, working principle, and regulation strategy of the adaptive control module will be described in detail below.

[0072] The adaptive control module consists of three core subsystems: a control instruction parser, a PID control system, and an execution feedback system. The control instruction parser is responsible for receiving the decision results output by the reinforcement learning model and converting them into standardized control target values; the PID control system is responsible for calculating specific control instructions based on the control target values and the current state; the execution feedback system is responsible for monitoring the execution effect and providing feedback information.

[0073] The control instruction parser is the front end of the adaptive control module, receiving the decision output of the reinforcement learning model, which is usually represented as a set of optimized parameters, such as the target cutter head rotation speed, thrust, and grouting pressure. The parser first performs a legality check on these parameters to ensure that they are within the allowable range of the equipment; then, according to the current construction status and historical experience, the parameters are fine-tuned and smoothed to avoid sudden changes in control instructions; finally, the adjusted parameters are converted into standardized control target values and transmitted to the PID control system.

[0074] The PID control system is the core of the adaptive control module, responsible for achieving precise control of the cutter head working parameters. This system adopts a multi-loop PID control architecture, including a cutter head rotation speed control loop, a thrust control loop, and a grouting control loop, etc. Each loop is managed by an independent PID controller, and the controllers maintain consistency through a coordination mechanism. Taking the cutter head rotation speed control as an example, the input of the PID controller is the error between the target rotation speed and the current actual rotation speed, and the output is the control signal for the cutter head drive system. The controller calculates the control quantity according to the three terms of proportional (P), integral (I), and derivative (D), and the formula is:

[0075]

[0076] where u(t) is the control signal, e(t) is the error signal, and K p 、K i 、K d are the proportional, integral, and derivative coefficients respectively.

[0077] In a preferred embodiment, the PID control system adopts an adaptive PID control algorithm, which can automatically adjust the PID parameters according to the system response characteristics. Specifically, the system evaluates the applicability of the current PID parameters by analyzing indicators such as overshoot, rise time, and settling time during the control process, and dynamically adjusts the values of Kp, Ki, and Kd according to the evaluation results. For example, when it is detected that the system response is too slow, the value of Kp will be appropriately increased; when it is detected that the system oscillates, the value of Kp will be appropriately decreased and the value of Kd will be increased. This adaptive mechanism enables the control system to automatically adapt to the system dynamic characteristics under different working conditions and maintain the best control effect.

[0078] At the same time, the PID control system also introduces model predictive control (MPC) elements, uses the dynamic model of the system to predict future states, and optimizes the control sequence accordingly. MPC takes into account the future behavior of the controlled object, can anticipate upcoming changes in advance, and has better predictability and stability compared to traditional PID. In implementation, the MPC controller maintains an optimization problem within a rolling time domain, solves it once in each control cycle to obtain the optimal control sequence, and applies the first control action of it.

[0079] The execution feedback system is the backend of the adaptive control module, responsible for monitoring the execution effect of control instructions and providing feedback information. This system includes multiple feedback mechanisms: real-time feedback, short-term evaluation, and long-term optimization. Real-time feedback directly monitors the execution of control instructions, such as the deviation between the actual rotation speed and the target rotation speed of the cutter head; short-term evaluation analyzes short-term indicators of the control effect, such as the change trend of the cutter head load; long-term optimization focuses on the impact of control strategies on the overall construction effect, such as energy consumption, efficiency, and safety indicators. On the one hand, this multi-level feedback information is used to adjust the PID control parameters, and on the other hand, it is also fed back to the reinforcement learning model as the basis for model optimization.

[0080] In terms of specific regulation strategies, the adaptive control module realizes multi-objective collaborative control. For example, in hard rock areas, the system will coordinate the cutter head rotation speed and thrust to maintain an appropriate cutting ratio (the ratio of rotation speed to thrust) to avoid excessive tool wear; in water-rich sand layers, the system will increase the grouting pressure and reduce the tunneling speed to prevent ground settlement and soil erosion; when passing under sensitive buildings, the system will adopt a fine regulation strategy to strictly control the ground settlement amount. These control strategies for specific scenarios are based on the decisions of the reinforcement learning model and the summary of historical experience, and can cope with various complex construction environments.

[0081] In addition, the adaptive control module also has the ability of fault detection and fault-tolerant control. By monitoring the change trends and mutual relationships of key parameters, the system can detect potential faults at an early stage; when it detects that some actuators are abnormal, the system will automatically adjust the control strategy to minimize the impact of the fault and ensure the continuity and safety of construction.

[0082] It should be noted that the adaptive control module is a bridge connecting intelligent decision-making and actual execution. Through advanced control algorithms and multi-level feedback mechanisms, it realizes the precise regulation of the working parameters of the shield cutter head. This module can not only faithfully execute the decisions of the reinforcement learning model but also make fine-tuning and optimization according to real-time situations to ensure the stability, safety, and efficiency of the control process. Through the collaborative work of the intelligent decision-making model and the adaptive control module, the system realizes the full-closed-loop intelligent regulation from perception to decision-making and then to control, greatly improving the intelligent level of shield tunneling construction.

[0083] S301: The real-time regulation of the working parameters of the shield cutter head by the adaptive control module includes: adjusting the addition ratio of the accelerator through a PID closed-loop control system; real-time monitoring of the rotation speed, thrust, and grouting pressure of the shield cutter head and making dynamic adjustments according to the actual construction environment; the double-fluid grouting system is equipped with conductivity sensors and viscosity sensors to provide real-time feedback on the slurry mixing state.

[0084] S302: The reinforcement learning intelligent decision-making model is trained based on a dynamic reward function, and the weight coefficients of the dynamic reward function are automatically switched to different preset parameter combinations according to the construction scenario: the first parameter combination is adopted in the normal tunneling stage; the second parameter combination is adopted when passing through sensitive areas; the third parameter combination is adopted in the water-rich sand layer stage.

[0085] S303: The method also includes system expansion steps: data exchange and collaboration with external systems through an open API interface; loading user-defined control strategies through an SDK toolkit to correct the construction parameters of the shield cutter head in real time; the open API interface supports the RESTful protocol and the MQTT protocol and is compatible with data access from BIM platforms, ground-penetrating radars, and sonar devices.

[0086] S304: The SDK toolkit is built with a visual strategy editor that allows users to define the following control modes by dragging and dropping components: settlement-sensitive mode: the settlement threshold is set in the range of ±1.0 - 2.0 mm; energy-saving priority mode: the energy consumption weight is higher than other weights; hard rock tunneling mode: the thrust upper limit ≥ 20 Mpa, and the cutter head rotation speed ≤ 1.2 rpm.

[0087] In summary, through intelligent control and real-time optimization, the present invention significantly improves the efficiency and safety of shield tunneling construction, and can achieve high-precision automatic adjustment in complex geological and dynamic construction environments. This system is particularly suitable for urban tunnel construction, underground pipeline construction, and other underground engineering projects, effectively reducing the risks that may occur during construction, improving the precision and quality of construction, and significantly reducing energy consumption and construction costs. In addition, the present invention has the characteristics of flexible system expansion and customization, can be customized according to different engineering requirements, supports collaborative work with various devices and external platforms, meets the increasingly complex engineering management needs, and has broad application prospects and market potential.

[0088] Example 2, referring to Figure 1 - Figure 4 , which is the second embodiment of the present invention, provides an intelligent control method for construction machinery based on multi-modal perception and reinforcement learning.

[0089] The construction machinery and equipment of the present invention mainly refers to shield machines. As an important equipment for tunnel excavation, shield machines are widely used in projects such as urban rail transit and underground pipelines. With the expansion of project scale and the complexity of construction environment, the traditional operation methods and control systems of shield machines are difficult to meet the requirements of high efficiency, intelligence, and safety in modern construction. Traditional shield machines usually rely on simple sensor feedback and preset control rules, and it is difficult to cope with the changing geological environment and real-time changes, resulting in problems such as low construction efficiency, high energy consumption, and poor quality.

[0090] To improve the precision and efficiency of shield tunneling construction, existing technologies have begun to adopt intelligent means, such as PID algorithm optimization control, but its adaptability is insufficient and there is a large room for optimization. With the development of big data, artificial intelligence, and machine learning, data-driven intelligent decision-making and execution systems have become the key to improving the intelligence level of shields.

[0091] However, existing intelligent systems for shield machines mostly rely on single data sources and simple decision-making models, and cannot achieve real-time multi-dimensional optimization in complex environments. At the same time, existing multi-objective optimization methods usually only target a single parameter, lack comprehensive consideration of efficiency, energy consumption, and safety, and have poor scalability, making it difficult to adapt to engineering requirements and work in coordination with other devices.

[0092] Therefore, this patent proposes an intelligent control method and system for construction machinery based on multi-modal perception and reinforcement learning. By integrating perception, decision-making, execution, and expansion modules, precise construction control and efficient management are achieved. This system supports real-time data collection, intelligent decision-making generation, dynamic adjustment control, and works in coordination with external systems (such as BIM platforms, cloud servers), thereby improving the intelligence, efficiency, and environmental protection benefits of shield operations.

[0093] The system collects real-time data through a multi-modal perception module, processes data and makes decisions through a local server, conducts global optimization and collaborative management through a cloud server, and finally transmits control instructions to the shield machine control system. Meanwhile, the system supports the integration of the BIM platform and customized development by developers, enhancing the flexibility and scalability of the system through the SDK interface. Through this overall architecture, the system realizes the integrated operation of perception, decision-making, execution, and expansion, improving the intelligent level and construction efficiency of shield tunneling.

[0094] First, the system collects real-time data through a multi-modal perception module (the sensor system at the head of the shield), which includes various sensors such as fiber optic sensors, cutterhead vibration sensors, electromagnetic wave radars, etc., for detecting data such as underground geological conditions, shield machine operation status, and soil layer properties. All these perceived data are collected in real-time and the real-time data is transmitted to the local server for further processing and analysis.

[0095] Furthermore, the collected real-time data is transmitted to the local server and processed through an embedded CNN-LSTM hybrid model. Through this model, the local server can fuse multi-source data collected by the perception module and generate decision-making instructions suitable for the operation of the shield machine on this basis. These instructions cover key control parameters such as cutterhead rotation speed, thrust, and grouting volume, and are dynamically adjusted according to geological conditions and construction status.

[0096] Furthermore, the processed and optimized decision-making instructions and other key information are uploaded to the cloud server through the API interface. The cloud server is responsible for storing historical data and receiving real-time data uploads to update and optimize the decision-making model in the system. Through data analysis and model training, the cloud server continuously optimizes the decision-making algorithm and construction strategy, providing global optimization support for the construction process of the shield machine.

[0097] Furthermore, the cloud server exchanges data with the BIM platform through the API interface. The BIM platform provides building information models and construction progress data, collaboratively managing all information during the entire shield tunneling process. Through the connection of the API, the cloud server can obtain and update the data of the BIM platform in real-time, providing the latest information and progress reports of the construction for project managers, thereby supporting rapid decision-making and adjustment.

[0098] Furthermore, the cloud server also provides interfaces and tools to developers through the SDK interface, supporting developers in conducting data analysis, model training, and strategy feedback. Developers can upload real-time data, receive feedback control instructions, and optimize the system performance through the SDK. The use of the SDK enables developers to customize the development and strategy adjustment of the system according to actual needs, further improving the flexibility and adaptability of the system.

[0099] Furthermore, the control instructions generated by the cloud server will be sent to the shield machine control system through the API interface. This system will adjust the operating parameters of the shield machine in real time according to the control instructions. The instructions include adjusting the cutter head rotation speed, propulsion force, grouting volume, etc., to ensure that the shield operation can be optimized according to the real-time geological and construction conditions.

[0100] Furthermore, after the shield machine control system executes the instructions, it starts to collect data, monitors the construction status in real time, and transmits the feedback data back to the cloud server. These data will be used for closed-loop control to further optimize the decision-making process. Through this feedback mechanism, the system can continuously adjust the shield operation process to ensure the accuracy and safety of the construction.

[0101] Multi-modal perception module: It consists of distributed optical fiber sensors, cutter head vibration sensors, and electromagnetic wave radars. The sensor data eliminates the shield vibration interference through the wavelet-Fourier joint noise reduction algorithm, and collects the formation strain, temperature, moisture content, and fracture distribution data within the range of 20 - 50 meters in front of the shield machine in real time;

[0102] Intelligent decision-making module: It is used to receive the data transmitted by the multi-modal perception module, process the data based on the reinforcement learning algorithm, and generate optimized decisions on the cutter head working parameters;

[0103] Adaptive control module: It is used to adjust the working parameters of the cutter head in real time according to the control instructions generated by the intelligent decision-making module, including cutter head rotation speed, propulsion force, grouting volume, etc.;

[0104] System expansion module: It provides an open API interface and an SDK toolkit, supports the access of third-party devices and user-defined control strategies, is used for data interaction with external platforms, and expands the functions of the system according to the requirements of external systems.

[0105] The wavelet-Fourier joint noise reduction algorithm specifically includes:

[0106] Perform wavelet packet decomposition on the cutter head vibration signal and extract the effective components in the frequency band of 0 - 500Hz;

[0107] Perform Fourier transform on the optical fiber strain signal and filter out the noise components (50 - 150Hz) with the same frequency as the shield vibration;

[0108] Input the noise-reduced data into the transfer federated reinforcement learning model.

[0109] The reinforcement learning model adopts a CNN-LSTM hybrid network architecture. The input data includes the optical fiber strain gradient matrix, the main frequency of the vibration spectrum and the harmonic energy ratio, and the fracture distribution feature vector of the electromagnetic wave radar. The output is a multi-objective optimization parameter combination of the cutter head rotation speed, thrust, and grouting pressure.

[0110] In the adaptive control module, the dual-liquid grouting system is equipped with a conductivity sensor and a viscosity sensor to provide real-time feedback on the slurry mixing state and adjust the proportion of the accelerating agent added (in the range of 1% to 5%) through PID closed-loop control.

[0111] The open API interface of the system expansion module supports RESTful protocol and MQTT protocol, is compatible with BIM platform, geological radar and sonar equipment data access, and data transmission uses TLS1.3 encryption and OAuth2.0 authentication.

[0112] The transfer reinforcement learning (MRL) reward function is:

[0113] R=αE Saving +βT advance -γS Ssettlement

[0114] Among them, E Saving is the energy saving rate, T advance is the excavation efficiency (m / h), S Ssettlement is the settlement excess limit value (mm), α, β, γ are weight coefficients.

[0115] The SDK toolkit has a built-in visual policy editor that allows users to define the following control modes by dragging and dropping components:

[0116] Subsidence sensitive mode: Subsidence threshold setting range: ±1.0-2.0mm;

[0117] Energy-saving priority mode: energy consumption weight α ≥ 0.5;

[0118] Hard rock tunneling mode: upper limit of thrust ≥ 20Mpa, cutter head speed ≤ 1.2rpm.

[0119] The dynamic reward function weight coefficient automatically switches according to the construction scenario:

[0120] Conventional excavation stage: α = 0.3, β = 0.5, γ = 0.2;

[0121] Underpass sensitive area: α = 0.1, β = 0.2, γ = 0.7;

[0122] Water-rich sand layer stage: α=0.4, β=0.3, γ=0.3.

[0123] The distributed optical fiber sensors, cutterhead vibration sensors and electromagnetic wave radars are used to collect the stratum strain, temperature, water content and crack distribution data within 20-50 meters in front of the shield machine in real time;

[0124] Perform wavelet-Fourier joint noise reduction processing on the collected raw data to eliminate shield vibration interference;

[0125] Input the multi-source data after noise reduction into the CNN-LSTM hybrid model for spatio-temporal feature fusion, and output the formation strength grade and risk prediction results;

[0126] Adjust the proportion of the accelerator through PID closed-loop control, and dynamically adjust the grouting frequency and pressure to optimize the diffusion effect of the grout;

[0127] Access the BIM platform and third-party device data through the open API interface, and use the SDK toolkit to load the user-defined control strategy to correct the construction parameters in real time.

[0128] Figure 1 This is the implementation method diagram of the present invention, and the specific steps are as follows:

[0129] Step 1: Collect the strain and temperature data of the formation through distributed fiber optic sensors. The cutter head vibration sensor is used to obtain the vibration spectrum data, and the electromagnetic wave radar is used to detect the distribution of underground fissures, and collect the data from different sensors in real time.

[0130] Step 2: Input the multi-source data after noise reduction into the CNN-LSTM hybrid model for spatio-temporal feature fusion, and output the formation strength grade and risk prediction results.

[0131] Step 3: Conduct spatial distribution analysis on the strain data collected by the fiber optic sensor through CNN (Convolutional Neural Network), use LSTM (Long Short-Term Memory Network) to model the temporal features of the vibration data, and combine the fissure data provided by the electromagnetic wave radar. After comprehensive processing, evaluate the formation risk, such as important indicators such as the probability of boulders and water content.

[0132] Step 4: Adopt a dynamic reward function to calculate the optimized value of the target parameters, use the transfer reinforcement learning (MRL) framework, and optimize multiple target parameters such as the cutter head rotation speed, thrust and grouting pressure of the shield machine through incremental parameter adjustment and KL divergence constraint to balance energy efficiency, efficiency and safety; the transfer reinforcement learning framework realizes cross-project generalization through a pre-trained model and incremental learning, and forms a difference from the multi-node collaborative training of federated learning.

[0133] Step 5: Based on the PID closed-loop control system, adjust the proportion of the accelerator in real time, and optimize the utilization rate and diffusion effect of the grout through the gradient pulse grouting technology (adjust the grouting frequency and pressure fluctuation) to ensure that the cutter head working parameters and ground surface settlement are controlled within the predetermined range.

[0134] Step 6: Access the BIM platform through the API interface, obtain the geological model data, use the SDK toolkit to load and execute the user-defined control strategy, and realize the flexible access and collaborative operation of the system to third-party devices and external strategies.

[0135] Preferably, in step 1, multi-sensor fusion technology is used, combined with data synchronization methods, to ensure the temporal and spatial consistency of data collected by different sensors, thereby improving the reliability and accuracy of the data. High-precision sensors are used to ensure accurate monitoring of the underground environment.

[0136] Furthermore, in step 2, the system performs noise reduction processing on the multi-source data collected, eliminating possible sensor errors or environmental interferences. The noise-reduced data will be input into the CNN-LSTM hybrid model. The convolutional neural network (CNN) is used to extract spatial features, and the long short-term memory network (LSTM) is used to model the time-series data. Through this hybrid model, the system can analyze and fuse various types of data, thereby evaluating the strength level and potential risks of the formation, generating specific prediction results, and providing a scientific basis for subsequent construction decisions.

[0137] Furthermore, in step 3, the system analyzes the spatial distribution of the strain data collected by the fiber optic sensor through CNN, identifies the change trends and pressure distributions of the soil layers, and reveals possible weak areas. LSTM models the time-series characteristics of the vibration data to help capture dynamic information related to time changes, such as changes in cutter head load and soil density. Combining with the fracture data of the electromagnetic wave radar, the system comprehensively processes this information, thereby evaluating the overall risks of the underground geology, including factors such as boulders, underground cavities, and moisture content, to ensure the safety during the construction process.

[0138] Further, in step 4, the system optimizes the working state of the shield machine through a dynamic reward function. The reward function is adjusted based on multiple indicators (such as construction progress, cutter head load, soil layer strength, etc.) fed back in real time, and the target parameters (such as cutter head rotation speed, thrust, grouting pressure, etc.) are optimized. Through the migration reinforcement learning (MRL) framework, the system continuously adjusts the strategy so that each operation can be optimized based on the experience of the previous round. Through KL divergence constraint, the system ensures maintaining a multi-objective balance during the optimization process, avoiding excessive adjustment of a certain parameter that may lead to losses in efficiency or safety, thereby achieving the best construction effect.

[0139] Further, in step 5, the combination of the PID and the reinforcement learning model can make full use of their respective advantages. The PID controller is responsible for local and real-time feedback regulation, while the reinforcement learning model makes intelligent decisions and optimizations at the global level. The reinforcement learning model analyzes long-term feedback (such as construction efficiency, energy consumption, geological conditions, etc.) and continuously optimizes the parameter settings of the PID control (such as adjusting the values of proportional P, integral I, and derivative D), enabling the PID controller to show higher adaptability and intelligence at different construction stages (such as hard rock and soft soil layers).

[0140] Furthermore, in step 6, the SDK toolkit provides a development interface that allows developers to upload custom control strategies. Developers can upload Python scripts through the SDK to define the settlement-sensitive mode. After the system parses the scripts, control instructions are generated and loaded into the system. This enables the system to flexibly support customized construction strategies and adapt to different project requirements. Through the API interface, the system can effectively cooperate with third-party devices (such as real-time sensors and construction machinery) and external strategies (such as environmental change warning systems) to achieve full automation and intelligence in the construction process.

[0141] Embodiment 3 is the third embodiment of the present invention. What is different from the previous two embodiments is:

[0142] If the above-mentioned functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0143] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0144] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing it as appropriate, and then storing it in a computer memory.

[0145] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0146] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. An intelligent control method for construction machinery based on multi-modal perception and reinforcement learning, characterized in that: Including: obtaining multi-modal perception data within the operation area of the shield machine; Inputting the multi-modal perception data into a preset reinforcement learning intelligent decision-making model; According to the output of the reinforcement learning intelligent decision-making model, the working parameters such as the rotation speed, thrust, and grouting volume of the shield cutter head are adjusted in real time through an adaptive control module.

2. The intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to claim 1, characterized in that: The multi-modal perception data includes: distributed optical fiber sensing data, cutter head vibration monitoring data, electromagnetic wave radar detection data, and temperature monitoring data; the multi-modal perception data is the formation strain, temperature, moisture content, and fracture distribution data within 20 - 50 meters in front of the shield machine collected in real time by the multi-modal perception module.

3. The intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to claim 2, characterized in that: The reinforcement learning intelligent decision-making model adopts a CNN-LSTM hybrid network architecture, which is used to predict the formation strength grade and risks and generate optimized decisions on the cutter head working parameters.

4. The intelligent control method for construction machinery based on multi-modal perception and reinforcement learning according to claim 3, characterized in that: The real-time adjustment of the working parameters of the shield cutter head by the adaptive control module includes: adjusting the addition ratio of the accelerating agent through a PID closed-loop control system; monitoring the rotation speed, thrust, and grouting pressure of the shield cutter head in real time and making dynamic adjustments according to the actual construction environment; the double-fluid grouting system is equipped with a conductivity sensor and a viscosity sensor to feedback the slurry mixing state in real time.

5. The intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to claim 4, characterized in that: The reinforcement learning intelligent decision-making model is trained based on a dynamic reward function, and the weight coefficients of the dynamic reward function are automatically switched to different preset parameter combinations according to the construction scenario: the first parameter combination is adopted in the normal tunneling stage; the second parameter combination is adopted when passing through sensitive areas; the third parameter combination is adopted in the water-rich sand layer stage.

6. The intelligent control method for construction machinery based on multi-modal perception and reinforcement learning according to claim 4, wherein: This method further includes system expansion steps: performing data exchange and collaboration with external systems through an open API interface; loading user-defined control strategies through an SDK toolkit to correct the construction parameters of the shield cutter head in real time; the open API interface supports the RESTful protocol and the MQTT protocol and is compatible with data access from BIM platforms, ground penetrating radars, and sonar devices.

7. The intelligent control method for construction machinery with multi-modal perception and reinforcement learning according to claim 4, characterized in that: The SDK toolkit is built with a visual strategy editor that allows users to define the following control modes by dragging and dropping components: settlement-sensitive mode: the settlement threshold setting range is ±1.0 - 2.0 mm; energy-saving priority mode: the energy consumption weight is higher than other weights; hard rock tunneling mode: the thrust upper limit ≥ 20 Mpa, and the cutter head rotation speed ≤ 1.2 rpm.

8. A system adopting an intelligent control method for construction machinery with multimodal perception and reinforcement learning as described in any one of claims 1 to 7, characterized in that: Including a data acquisition module that obtains multi-modal perception data within the operation area of the shield machine; A processing module that inputs the multi-modal perception data into a preset reinforcement learning intelligent decision-making model; A regulation module that, according to the output of the reinforcement learning intelligent decision-making model, adjusts the working parameters such as the rotation speed, thrust, and grouting volume of the shield cutter head in real time through an adaptive control module.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it realizes the steps of a multi-modal perception and reinforcement learning-based intelligent regulation method for construction machinery according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it realizes the steps of a multi-modal perception and reinforcement learning-based intelligent regulation method for construction machinery according to any one of claims 1 to 7.

Images