Tunnel boring machine excavation parameter prediction method based on offline reinforcement learning and device

Abstract

The present disclosure provides a tunnel boring machine excavation parameter prediction method based on offline reinforcement learning and a device. The method comprises: acquiring historical operating data generated by a tunnel boring machine within a specified time interval under normal operating conditions, and preprocessing the data; constructing a tunnel boring machine excavation parameter prediction model on the basis of an offline reinforcement learning model, and training the tunnel boring machine excavation parameter prediction model by means of historical point data in the preprocessed historical operating data; and inputting point data generated by the tunnel boring machine during real-time operation into the trained tunnel boring machine excavation parameter prediction model, and outputting a predicted value of a tunnel boring machine excavation parameter to guide the excavation of the tunnel boring machine. By means of the present invention, a corresponding excavation parameter can be predicted on the basis of point data generated by a tunnel boring machine during operation, providing guidance for a tunnel boring machine operator.

Description

A shield tunneling parameter prediction method and device based on offline reinforcement learning TECHNICAL FIELD

[0001] The present application relates to the technical field of shield tunneling, and in particular to a shield tunneling parameter prediction method and device based on offline reinforcement learning, a shield tunneling parameter prediction equipment and a storage medium. BACKGROUND

[0002] A shield tunneling machine is a tunnel boring machine using the shield method. The shield construction method is to build (lay) the "shield" (referring to the supporting segment) of the tunnel while the tunneling machine is excavating, which is different from the open construction method. In the international community, the broad sense of shield tunneling machine can also be used in rock strata, while in China, the tunnel boring machine used in soft soil strata is commonly referred to as a shield tunneling machine.

[0003] It is understood that the excavation amount using the shield method accounts for 45% of the total amount of underground engineering construction. Although the shield tunneling machine is expensive, it can increase the efficiency of underground excavation by 8 to 10 times, and in the construction process, there is no need for large-scale demolition on the ground, no traffic interruption, no noise during construction, no ground subsidence, and no impact on the normal life of residents. However, large shield tunneling machines have high technical added value and complex manufacturing processes, and only a few enterprises in Europe, the United States and Japan can research and produce them.

[0004] Tunnel construction using shield method machinery has the characteristics of high automation, labor saving, fast construction speed, one-time hole forming, no influence of weather, control of ground settlement during excavation, reduction of influence on ground buildings, and no influence on ground traffic during underwater excavation. In the case of long tunnel line and large burial depth, the use of shield tunneling machines is more economical and reasonable. However, the operation of the shield tunneling machine should be stable and accurate during operation, and any incorrect operation by the operator during operation should be avoided to prevent significant losses. However, it is difficult for the operator to avoid incorrect operation during work.

[0005] SUMMARY

[0006] The present application provides a shield tunneling parameter prediction method and device based on offline reinforcement learning, equipment and storage medium, which aims to predict the corresponding tunneling parameters according to the point data in the operation process of the shield tunneling machine, and to provide guidance for the operator of the shield tunneling machine;

[0007] Therefore, the purpose of the present application is to provide a shield tunneling parameter prediction method based on offline reinforcement learning, comprising:

[0008] Obtain historical operation data of the shield tunneling machine within a specified time interval under normal operation conditions, and perform data preprocessing;

[0009] Based on an offline reinforcement learning model, a tunneling parameter prediction model of a shield tunneling machine is constructed, and the tunneling parameter prediction model of the shield tunneling machine is trained through historical point data in historical running data after preprocessing;

[0010] The point data of the real-time running of the shield tunneling machine is input into the trained tunneling parameter prediction model of the shield tunneling machine, and a predicted value of the tunneling parameter of the shield tunneling machine is output, to guide the tunneling of the shield tunneling machine.

[0011] The historical running data of the shield tunneling machine in a specified time interval under normal running conditions is obtained, including:

[0012] A shield tunneling machine simulator is constructed to simulate the tunneling work of the shield tunneling machine in a preset working scene;

[0013] Point data generated when a real shield tunneling machine performs tunneling work in the same working scene is obtained from a database;

[0014] The point data is input into the shield tunneling machine simulator to generate tunneling parameters corresponding to the point data;

[0015] The point data and the corresponding tunneling parameters are integrated to form the historical running data of the shield tunneling machine in a specified time interval under normal running conditions.

[0016] The historical running data is preprocessed, the completeness and quality of the data set are improved through data imputation and data augmentation, and the performance of the shield tunneling machine simulator is improved.

[0017] The data imputation method type at least includes:

[0018] Mean imputation: using the mean value of the data to fill in the missing values, suitable for numerical features;

[0019] Median imputation: using the median value of the data to fill in the missing values, suitable for numerical features, and robust to outliers;

[0020] Mode imputation: using the mode of the data to fill in the missing values, suitable for classification or discrete features;

[0021] Interpolation method: according to the relationship between the existing data points, the value of the missing data is inferred; the interpolation method type at least includes linear interpolation, polynomial interpolation, and spline interpolation;

[0022] Nearest Neighbor Imputation: Fill in missing values using the values of nearest neighbors based on similarity between samples; where Euclidean distance or Manhattan distance is used to measure the distance between samples;

[0023] Regression Imputation: Use other features as independent variables to build a regression model to predict missing values;

[0024] Random Forest Imputation: Use random forest algorithm to predict missing values based on the values of other features.

[0025] Among them, the data augmentation method type at least includes:

[0026] Data mixing: combining, superimposing or mixing different categories of samples to generate new samples;

[0027] Adding noise: adding Gaussian noise or random interference noise to the data to simulate real-world environmental changes and disturbances;

[0028] Generative Adversarial Network (GAN): Use generative adversarial network to generate synthetic data to increase the diversity and quantity of the data set.

[0029] Among them, the data type of the point data at least includes: average pushing speed, total pushing force of the pushing cylinder, cutter head speed, penetration, screw machine speed, soil tank soil pressure, cutter head torque and screw machine torque;

[0030] The data type of the tunneling parameter at least includes: average pushing speed, cutter head speed, screw machine speed.

[0031] Among them, the data type of the soil tank soil pressure includes: upper soil tank soil pressure, right upper soil tank soil pressure, right lower soil tank soil pressure, lower soil tank soil pressure, left lower soil tank soil pressure and left upper soil tank soil pressure.

[0032] In addition, the purpose of the present application is also to provide a shield tunneling parameter prediction device based on offline reinforcement learning, comprising:

[0033] A data acquisition module is configured to acquire historical operation data of the shield tunneling machine within a specified time interval under normal operation conditions and perform data preprocessing;

[0034] A model construction module is configured to construct a shield tunneling parameter prediction model based on an offline reinforcement learning model and train the shield tunneling parameter prediction model using historical point data in the preprocessed historical operation data.

[0035] The tunneling parameter prediction module is configured to input point data of real-time running of the shield tunneling machine into the trained shield tunneling machine tunneling parameter prediction model, and output a predicted value of the tunneling parameter of the shield tunneling machine, so as to guide the tunneling of the shield tunneling machine.

[0036] The application also provides a computer device, a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of the preceding technical solutions when executing the computer program.

[0037] The application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executable on the processor to implement the method according to the preceding technical solution.

[0038] Different from the prior art, the shield tunneling machine tunneling parameter prediction method based on offline reinforcement learning provided by the application comprises the following steps: obtaining historical running data of a shield tunneling machine in a specified time interval under normal running conditions, and performing data preprocessing; constructing a shield tunneling machine tunneling parameter prediction model based on an offline reinforcement learning model, and training the shield tunneling machine tunneling parameter prediction model by using historical point data in the historical running data after preprocessing; inputting point data of real-time running of the shield tunneling machine into the trained shield tunneling machine tunneling parameter prediction model, and outputting a predicted value of the tunneling parameter of the shield tunneling machine, so as to guide the tunneling of the shield tunneling machine. According to the application, the corresponding tunneling parameter can be predicted according to the point data in the running process of the shield tunneling machine, thereby providing guidance for the work of the operator of the shield tunneling machine. BRIEF DESCRIPTION OF DRAWINGS

[0039] The above and / or additional aspects and advantages of the application will become apparent and be readily understood from the following description, taken in conjunction with the drawings, in which:

[0040] Fig. 1 is a flowchart of a shield tunneling machine tunneling parameter prediction method based on offline reinforcement learning provided by the application.

[0041] Fig. 2 is a structural diagram of a shield tunneling machine tunneling parameter prediction device based on offline reinforcement learning provided by the application.

[0042] Fig. 3 is a structural diagram of a non-transitory computer readable storage medium storing computer instructions provided by the application. DETAILED DESCRIPTION

[0043] Embodiments of the application are described in detail below with reference to the accompanying drawings, in which like reference numerals refer to like elements or elements having the same or similar function throughout. The embodiments described below are exemplary and are intended to explain the application, and cannot be understood as limiting the application.

[0044] An off-line reinforcement learning based tunneling parameter prediction method for a shield tunneling machine is described below with reference to the accompanying drawings.

[0045] Fig. 1 is a flowchart of an off-line reinforcement learning based tunneling parameter prediction method for a shield tunneling machine according to an embodiment of the present application. The implementation steps of the method are as follows:

[0046] Step 110: Obtain historical operation data of the shield tunneling machine within a specified time interval under normal operation conditions and perform data preprocessing.

[0047] Specifically, the historical operation data of the shield tunneling machine within a specified time interval under normal operation conditions obtained in the present application includes:

[0048] A shield tunneling machine simulator is constructed to simulate the tunneling work of the shield tunneling machine in a preset working scenario;

[0049] To better fit the actual working conditions, the most similar records to the predicted values calculated by the model are found from the database storing the historical data on site and are selected as the final predicted values. Since the model predicted values are multi-dimensional vectors and the magnitudes of the dimensions differ greatly, a reasonable method for calculating the similarity of two vectors is designed. Since the present application focuses on the overall similarity of two vectors, i.e., each dimension should be as similar as possible, the Manhattan distance, which is not sensitive to the magnitude difference, is used to calculate the similarity, i.e., the sum of the absolute values of the dimension differences. In addition, the two vectors are normalized before calculating the Manhattan distance, further reducing the influence of the magnitude on the overall similarity.

[0050] Obtain point data generated by the real shield tunneling machine when performing tunneling work in the same working scenario from the database;

[0051] Input the point data into the shield tunneling machine simulator to generate tunneling parameters corresponding to the point data;

[0052] Integrate the point data and the corresponding tunneling parameters to form the historical operation data of the shield tunneling machine within a specified time interval under normal operation conditions.

[0053] The data types of the point data at least include: average pushing speed, total pushing cylinder thrust, cutter head speed, penetration, screw machine speed, soil tank soil pressure, cutter head torque, and screw machine torque.

[0054] The data types of the tunneling parameters at least include: average pushing speed, cutter head speed, and screw machine speed.

[0055] The data types of the soil tank soil pressure include: upper soil tank soil pressure, right upper soil tank soil pressure, right lower soil tank soil pressure, lower soil tank soil pressure, left lower soil tank soil pressure, and left upper soil tank soil pressure.

[0056] Data preprocessing is performed on historical operation data to improve the integrity and quality of the data set through data imputation and data augmentation, thereby improving the performance of the shield machine simulator.

[0057] The data imputation method types include at least:

[0058] Mean imputation: using the mean value of the data to fill in the missing values, suitable for numerical features;

[0059] Median imputation: using the median value of the data to fill in the missing values, suitable for numerical features, and robust to outliers;

[0060] Mode imputation: using the mode of the data to fill in the missing values, suitable for categorical or discrete features;

[0061] Interpolation methods: according to the relationship between existing data points, the missing data values are estimated; the interpolation method types include at least linear interpolation, polynomial interpolation, and spline interpolation;

[0062] Nearest neighbor imputation: according to the similarity between samples, the value of the nearest neighbor sample is used to fill in the missing values; the Euclidean distance or Manhattan distance is used to measure the distance between samples;

[0063] Regression imputation: using other features as independent variables to establish a regression model to predict missing values;

[0064] Random forest imputation: using random forest algorithm to predict missing values according to the values of other features.

[0065] The data augmentation method types include at least:

[0066] Data mixing: combining, superimposing or mixing different categories of samples to generate new samples;

[0067] Adding noise: adding Gaussian noise or random interference noise to the data to simulate real-world environmental changes and disturbances;

[0068] Generative adversarial network (GAN): using a generative adversarial network to generate synthetic data to increase the diversity and quantity of the data set.

[0069] Step 120: based on the offline reinforcement learning model, a shield tunneling parameter prediction model is constructed, and the shield tunneling parameter prediction model is trained through historical point position data in the preprocessed historical running data.

[0070] In this step, one of the key goals of the model-based prediction is to significantly improve the tunneling speed. In the case where most of the tunneling speeds in the training data are at low speed, it is difficult to achieve the goal only by relying on reinforcement learning, or the model is easily trapped in a local optimal solution during the training process, which is manifested as a very aggressive prediction value given each time. The method of the present application sorts these vectors in descending order with tunneling speed as the key value, and finds a vector with larger tunneling speed and closer value compared with the previous tunneling state as the prediction value of this time. Increasing this screening mechanism can guide the reinforcement learning algorithm to calculate a tunneling speed that is significantly improved and has a smooth transition.

[0071] The present application uses the historical data of the previous N rings of the current ring as the training set for each prediction, making full use of the prior data closest to the current ring and the stratum characteristics most similar to the current ring, instead of the idea of using all line historical data as the training set.

[0072] Step 130: input the point position data of the real-time running of the shield into the trained shield tunneling parameter prediction model, output the prediction value of the shield tunneling parameter, and use it to guide the shield tunneling.

[0073] In the actual shield working process, after determining the point position data based on the actual working condition, the point position data is input into the model trained by the present application, and the tunneling parameter corresponding to the point position data can be output based on the model. Further, the shield operator can operate the shield according to the tunneling parameter.

[0074] As shown in FIG. 2, the present application also provides a shield tunneling parameter prediction device 200 based on offline reinforcement learning, comprising:

[0075] The data acquisition module 210 is configured to acquire historical running data of the shield within a specified time interval under normal running conditions, and perform data preprocessing.

[0076] The model construction module 220 is configured to construct a shield tunneling parameter prediction model based on an offline reinforcement learning model, and train the shield tunneling parameter prediction model through historical point position data in the preprocessed historical running data.

[0077] The tunneling parameter prediction module 230 is configured to input the point position data of the real-time running of the shield into the trained shield tunneling parameter prediction model, output the prediction value of the shield tunneling parameter, and use it to guide the shield tunneling.

[0078] As shown in FIG. 3, the non-transitory computer-readable storage medium includes a memory 810 storing instructions, an interface 830, and the instructions can be executed by a shield tunneling parameter prediction processor 820 based on offline reinforcement learning to complete the method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0079] In order to achieve the embodiments, the application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the shield tunneling parameter prediction based on offline reinforcement learning according to the embodiments of the application.

[0080] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the application. In the description of the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0081] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0082] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions (or steps) in the process, and the various embodiments of the application include additional implementations in which the order of steps can differ from those shown or discussed, including a step can occur at other times, including as recited in the description, and additional functions can be added or performed at a similar time. It is also possible to implement a process or method in code using functions, a portion of which can be used, reused, performed, or invoked in each specific implementation.

[0083] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of instructions to implement logic functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a machine-readable storage device (e.g., magnetic, optical or other) a machine-readable storage diskette (e.g., floppy, flexible or other), a machine-readable storage card (e.g., ROM, EEPROM, flash memory or other), a machine- readable storage tape (e.g., magnetic, optical or other), a machine-readable storage medium (e.g., a portable electronic device, a computer diskette, a computer memory, a broadcast transmission, or the like), or a machine-readable interface device (e.g., a wireless link, optical link or other). The computer-readable medium can also be, or be included in, a computer program product apparatus that tangibly embodies the programming of instructions. The instructions can be executable by a processor of the instruction execution system, apparatus, or device. In another embodiment, the logic and / or steps represented in flow diagrams or otherwise described herein can be considered as a sequence of logic elements, such as steps, functions, operations, or the like, that can be embodied in any computer-readable medium for execution by an instruction execution system, apparatus, or device.

[0084] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. As such, each step or method can be embodied by computer-readable instructions arranged to execute on the instruction execution system, apparatus, or device, or in combinations thereof. In another embodiment, the steps or methods can be implemented using a combination of any of the following technologies, hardware, firmware, and / or software, or combinations thereof, as known to those skilled in the art.

[0085] Those of skill in the art would understand that information and signals can be represented using any of a variety of technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0086] In addition, each function unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0087] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A shield tunneling parameter prediction method based on offline reinforcement learning, characterized in that, The method comprises the following steps: acquiring historical operation data of a shield tunneling machine in a specified time interval under normal operation conditions, and performing data preprocessing; constructing a shield tunneling parameter prediction model based on an offline reinforcement learning model, and training the shield tunneling parameter prediction model by using historical point data in the historical operation data after preprocessing; inputting point data in real-time operation of the shield tunneling machine into the trained shield tunneling parameter prediction model, and outputting a predicted value of a shield tunneling parameter to guide the shield tunneling machine.

2. The shield tunneling parameter prediction method based on offline reinforcement learning according to claim 1, characterized in that, The step of acquiring historical operation data of a shield tunneling machine in a specified time interval under normal operation conditions comprises the following steps: constructing a shield tunneling machine simulator to simulate the tunneling work of the shield tunneling machine in a preset working scenario; acquiring point data generated by a real shield tunneling machine when performing tunneling work in the same working scenario from a database; inputting the point data into the shield tunneling machine simulator to generate tunneling parameters corresponding to the point data; integrating the point data and the corresponding tunneling parameters to form historical operation data of the shield tunneling machine in a specified time interval under normal operation conditions.

3. The shield tunneling parameter prediction method based on offline reinforcement learning according to claim 1, characterized in that, The historical operation data is preprocessed to improve the integrity and quality of the data set by data imputation and data augmentation, and to improve the performance of the shield tunneling machine simulator.

4. The shield tunneling parameter prediction method based on offline reinforcement learning according to claim 3, characterized in that, The data imputation method types at least include: Mean Imputation: using the mean value of data to fill in missing values, suitable for numerical features; Median Imputation: using the median value of data to fill in missing values, suitable for numerical features, and robust to outliers; Mode Imputation: using the mode value of data to fill in missing values, suitable for classification or discrete features; Interpolation Methods: according to the relationship between existing data points, the value of missing data is estimated; wherein, the interpolation method types at least include linear interpolation, polynomial interpolation, and spline interpolation; Nearest Neighbor Imputation: according to the similarity between samples, the value of the nearest neighbor sample is used to fill in the missing value; wherein, the Euclidean distance or Manhattan distance is used to measure the distance between samples; Regression Imputation: using other features as independent variables to establish a regression model to predict missing values; Random Forest Imputation: using random forest algorithm to predict missing values according to the values of other features.

5. The shield tunneling parameter prediction method based on offline reinforcement learning according to claim 3, characterized in that, The data augmentation method types at least include: Data Mixing: combining, superimposing or mixing different categories of samples to generate new samples; Adding noise: adding Gaussian noise or random interference noise to the data to simulate environmental changes and disturbances in the real world; Generative Adversarial Network (GAN): using a generative adversarial network to generate synthetic data to increase the diversity and quantity of the data set.

6. The shield tunneling parameter prediction method based on offline reinforcement learning according to claim 1, characterized in that, The data type of the point position data at least includes: average pushing speed, total pushing cylinder thrust, cutter head rotating speed, penetration, screw rotating speed, earth chamber earth pressure, cutter head torque and screw torque. The data type of the tunneling parameter at least includes: average pushing speed, cutter head rotating speed and screw rotating speed.

7. The shield tunneling parameter prediction method based on offline reinforcement learning according to claim 6, characterized in that, The data type of the earth chamber earth pressure includes: upper earth chamber earth pressure, right upper earth chamber earth pressure, right lower earth chamber earth pressure, lower earth chamber earth pressure, left lower earth chamber earth pressure and left upper earth chamber earth pressure.

8. A shield tunneling parameter prediction device based on offline reinforcement learning, characterized in that, The method comprises: a data acquisition module, configured to acquire historical running data of a shield machine in a specified time interval under a normal running condition, and perform data preprocessing; a model construction module, configured to construct a shield machine tunneling parameter prediction model based on an offline reinforcement learning model, and train the shield machine tunneling parameter prediction model by historical point position data in the historical running data after preprocessing; a tunneling parameter prediction module, configured to input point position data when the shield machine is running in real time into the trained shield machine tunneling parameter prediction model, and output a predicted value of a shield machine tunneling parameter, so as to guide shield machine tunneling.

9. A computer device, comprising: The computer program is executed by the processor to implement the method in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method in any one of claims 1-7.

Images