A peripheral eye blasting scheme generation method based on deep reinforcement learning

Abstract

The application is suitable for the technical field of tunnel blasting, and provides a peripheral hole blasting scheme generation method based on deep reinforcement learning, comprising: training a strategy network by using deep reinforcement learning to obtain an optimal strategy network; a reward function during the training of the strategy network is calculated based on a local predicted blasting effect output by a three-hole blasting prediction model and an overall predicted blasting effect output by a smooth blasting effect prediction model; a blasting scheme and a blasting effect of a blasted area are obtained; the blasted area and a target area have the same rock mass conditions; an input vector is constructed based on the blasting scheme and the blasting effect of the blasted area, and the constructed input vector is processed by using the optimal strategy network to obtain a peripheral hole blasting scheme of the target area. The application can improve the blasting precision of the peripheral hole.

Description

Technical Field

[0001] This application belongs to the field of tunnel blasting technology, and in particular relates to a method for generating peripheral eye blasting schemes based on deep reinforcement learning. Background Technology

[0002] Drill-and-blast method, as the mainstream construction method for hard rock tunnel excavation, has significant advantages such as low cost, high construction efficiency, and fast excavation speed. However, in actual blasting construction, the geological strata often exhibit abrupt changes, with some strata containing hard rock masses or weak interlayers. Traditional blasting schemes lack adaptive optimization capabilities, leading to severe over- and under-excavation during blasting, and even risks such as collapses. Therefore, the refined construction concept of "one blast, one design" has gradually become an industry consensus. Existing blasting scheme designs typically require obtaining various parameters of the rock mass on-site, with engineers designing based on experience. However, parameters such as borehole spacing and charge quantity are difficult to adjust differentiated according to local geological conditions, easily resulting in overall blasting effects but severe over- or under-excavation in local areas, thus affecting tunnel formation quality and subsequent construction progress. Furthermore, geological parameters are often very complex, and surveying and marking work consumes a lot of manpower and resources, seriously affecting the normal construction cycle. With the rapid development of computer technology and artificial intelligence in recent years, blasting construction scheme design should move towards intelligentization.

[0003] Currently, deep reinforcement learning, as a common autonomous learning method, obtains better decision results through repeated interaction with the environment and has been widely applied to various engineering decision-making problems. However, for the problem of generating peripheral eye blasting schemes in the field of blasting engineering, due to the complex and variable geological conditions and the lack of a unified representation method for blasting schemes, deep reinforcement learning methods are difficult to construct effective input vectors when applied to scheme optimization. Furthermore, the use of uniform parameters and the neglect of local rock layer differences in blasting scheme design ultimately result in low blasting accuracy for peripheral eyes. Summary of the Invention

[0004] This application provides a method for generating peripheral eye blasting schemes based on deep reinforcement learning, which can solve the problem of low blasting accuracy of peripheral eyes.

[0005] This application provides a method for generating peripheral eye explosion schemes based on deep reinforcement learning, including: Step 1: Train the policy network using deep reinforcement learning to obtain the optimal policy network. The reward function for training the policy network is calculated based on the local predicted blasting effect output by the three-hole blasting prediction model and the overall predicted blasting effect output by the smooth blasting effect prediction model. The smooth blasting effect prediction model is used to predict the effect of the blasting scheme output by the policy network and outputs the overall predicted blasting effect. The three-hole blasting prediction model is used to predict the effect of a part of the blasting scheme output by the policy network and outputs the local predicted blasting effect. Step 2: Obtain the blasting plan and blasting effect of the blasted area; the blasted area and the target area have the same rock mass conditions; Step 3: Construct an input vector based on the blasting scheme and blasting effect of the blasted area, and process the constructed input vector using the optimal policy network to obtain the peripheral eye blasting scheme of the target area. The peripheral eye blasting scheme of the target area includes the blasting design parameters of multiple peripheral eyes in the target area. The blasting design parameters of multiple peripheral eyes are generated one by one by the optimal policy network. The optimal policy network obtains the blasting design parameters of multiple peripheral eyes based on multiple distinct input vectors. The multiple input vectors correspond one-to-one with the blasting design parameters of multiple peripheral eyes.

[0006] Optionally, an input vector is constructed based on the blasting scheme and blasting effect of the blasted area, and the constructed input vector is processed using an optimal policy network to obtain the perimeter eye blasting scheme of the target area, including: Step 3.1: Use the blasting scheme and blasting effect of the blasted area as the input vector of the optimal strategy network; Step 3.2: Process the input vector using the optimal policy network to generate a blasting design parameter for the peripheral eye, and determine whether the blast hole generation stop condition is met. If it is met, use the generated blasting design parameter for the peripheral eye as the blasting scheme for the peripheral eye of the target area; otherwise, proceed to step 3.3. Step 3.3: Add the generated blasting design parameters of the peripheral eye to the input vector, use the input vector with the generated blasting design parameters of the peripheral eye as the input vector in step 3.2, and return to execute step 3.2.

[0007] Optionally, the blasting scheme for the blasted area includes the blasting design parameters for each peripheral hole in the blasted area, and the blasting effect of the blasted area includes: the blasting effect of each peripheral hole in the blasted area, the overall over-excavation volume of the blasted area, the overall under-excavation volume of the blasted area, and the maximum over-excavation value of the overall blasting.

[0008] Optional blasting design parameters include the thickness of the blasting layer, the spacing between boreholes, the charge length, the decoupling coefficient, and the charge structure.

[0009] Optionally, the condition for stopping borehole generation is: the absolute value of the difference between the sum of the borehole spacing of all peripheral holes in the current blasting scheme and the sum of the borehole spacing of all peripheral holes in the previous blasting scheme is less than a preset value. The current blasting scheme includes: the blasting design parameters of all peripheral holes generated up to the current moment; the previous blasting scheme includes: the blasting design parameters of all peripheral holes in the current blasting scheme except for the blasting design parameters of the last generated peripheral hole.

[0010] Optionally, the three-hole blasting prediction model is obtained by training the regression prediction network using a simulation training dataset. Each training sample in the simulation training dataset consists of blasting data under two identical rock mass conditions. The blasting data includes: blasting design parameters of a blasting group consisting of three adjacent peripheral holes, and the blasting effect of the central peripheral hole of the blasting group.

[0011] Optionally, the smooth blasting effect prediction model is obtained by training a network model with a self-attention mechanism using an actual blasting training dataset. Each training dataset in the actual blasting training dataset consists of two blasting samples under the same rock mass conditions. The blasting samples include: the perimeter hole blasting scheme of the actual blasted area and the overall blasting effect of the actual blasted area.

[0012] Optionally, the reinforcement learning training dataset for training the policy network may include: blasting design parameters for each peripheral eye within the historical blasting area, and blasting effects for each peripheral eye within the historical blasting area.

[0013] Optionally, during the training of the policy network, the blasting scheme output by the policy network includes: blasting design parameters for multiple peripheral holes in the training area that have the same rock mass conditions as the historical blasting areas; the blasting design parameters for multiple peripheral holes in the training area are generated one by one; the method for generating the peripheral hole blasting scheme also includes: During the training of the policy network, for the blasting scheme output by the policy network, according to the generation order of the blasting design parameters of the peripheral eyes, the three peripheral eyes corresponding to every three adjacent blasting design parameters are taken as a blasting unit.

[0014] Optional, reward function for: ; in, Indicates the number of explosive units. This represents the reduction factor. Indicates explosive unit Corresponding localized explosion effect rewards, This indicates the overall explosion effect reward; ; This indicates that the three-hole blasting prediction model is effective for blasting units. The local predicted explosion effect output during effect prediction. Indicates the effect of blasting on a local target; ; This represents the overall predicted blasting effect output by the smooth blasting effect prediction model when predicting the effect of the blasting scheme output by the policy network. This indicates the overall effect of the explosion on the target.

[0015] The above-mentioned solution in this application has the following beneficial effects: In the embodiments of this application, a policy network is trained using deep reinforcement learning to obtain an optimal policy network. Then, blasting schemes and blasting effects of blasted areas with the same rock mass conditions as the target area are acquired. An input vector is constructed based on these blasting schemes and effects. Finally, the optimal policy network processes the constructed input vector to obtain the peripheral eye blasting scheme for the target area. Since the optimal policy network generates peripheral eye blasting schemes one by one, and each peripheral eye corresponds to a different input vector, it effectively avoids the problem of using uniform parameters and ignoring local rock strata differences in existing blasting scheme designs. This effectively prevents over-excavation and under-excavation phenomena caused by weak interlayers or hard rocks in the rock strata, providing an effective technical means for the scientific optimization of blasting schemes. This allows for improved blasting accuracy of peripheral eyes when using the peripheral eye blasting scheme generated in this application.

[0016] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a peripheral eye blasting scheme generation method based on deep reinforcement learning provided in an embodiment of this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0021] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] To address the low blasting accuracy of peripheral eye blasting, this application provides a method for generating peripheral eye blasting schemes based on deep reinforcement learning. This method trains a policy network using deep reinforcement learning to obtain an optimal policy network. Then, it acquires blasting schemes and blasting effects from blasted areas with the same rock mass conditions as the target area. An input vector is constructed based on these blasting schemes and effects. Finally, the optimal policy network processes the constructed input vector to obtain the peripheral eye blasting scheme for the target area. Since the optimal policy network generates peripheral eye blasting schemes one by one, and each peripheral eye corresponds to a different input vector, it effectively avoids the problem of using uniform parameters and ignoring local rock strata differences in existing blasting scheme designs. This effectively prevents over-excavation and under-excavation phenomena caused by weak interlayers or hard rocks in the rock strata, providing an effective technical means for the scientific optimization of blasting schemes. This allows for a significant improvement in the blasting accuracy of peripheral eyes when using the peripheral eye blasting scheme generated by this application.

[0026] The peripheral eye explosion scheme generation method based on deep reinforcement learning provided in this application will be illustrated below with specific embodiments.

[0027] like Figure 1 As shown in the embodiments of this application, the peripheral eye explosion scheme generation method based on deep reinforcement learning includes the following steps: Step 1: Train the policy network using deep reinforcement learning to obtain the optimal policy network.

[0028] In some embodiments of this application, the reinforcement learning training dataset for training the policy network includes: blasting design parameters for each peripheral eye within a historical blasting area, and blasting effects for each peripheral eye within the historical blasting area. The historical blasting area is the area where peripheral eye blasting has actually been implemented.

[0029] The blasting design parameters include the thickness of the blasting layer, the spacing between boreholes, the charge length, the decoupling coefficient, and the charge structure (i.e., the location of the charge, such as charging at the bottom of the peripheral hole, charging in the middle of the peripheral hole, or charging at the top of the peripheral hole; here, the charge refers to the use of explosives).

[0030] The acquisition process of the aforementioned reinforcement learning training dataset is as follows: Technicians carry out blasting according to the pre-set blasting design parameters for each peripheral hole; after the blasting is completed on-site (i.e., in the historical blasting area), technicians should promptly disturb the rock mass in the blasting area (i.e., the historical blasting area) to cause the damaged rock mass to detach from the rock wall under the disturbance. Subsequently, a laser profiler is used to scan the tunnel cross-section to obtain and record the over-excavation volume, under-excavation volume, and maximum linear over-excavation length (i.e., the maximum over-excavation value) generated under the current blasting scheme. Then, in conjunction with the current blasting scheme, the blasting effect of each borehole (i.e., each peripheral hole in the historical blasting area) is calculated and recorded one by one. The blasting effect of each peripheral hole includes the over-excavation volume per unit length, the under-excavation volume per unit length, and the maximum linear over-excavation length corresponding to that peripheral hole.

[0031] The over-excavation volume per unit length, the under-excavation volume per unit length, and the maximum linear over-excavation length are used to characterize the rock blasting situation near a single blast hole (i.e., a peripheral hole), reflecting the physical information of the rock mass surrounding that blast hole. Specifically, the over-excavation volume per unit length refers to the over-excavation volume within the interval from half the left spacing to half the right spacing of the blast hole, divided by the total length of that interval; the under-excavation volume per unit length is calculated in the same way, i.e., the under-excavation volume within that interval divided by the total length of the interval; and the maximum linear over-excavation length refers to the maximum value of the linear over-excavation length within that interval.

[0032] During the training of the policy network, the blasting schemes output by the policy network include blasting design parameters for multiple peripheral holes in a training area that have the same rock mass conditions as historical blasting areas. The blasting design parameters for the multiple peripheral holes in the training area are generated one by one.

[0033] In some embodiments of this application, the aforementioned rock mass conditions include: rock mass compressive strength, rock mass tensile strength, rock mass density, and rock mass shear strength. It is understood that identical rock mass conditions refer to the differences between corresponding indicators (i.e., rock mass compressive strength, rock mass tensile strength, rock mass density, and rock mass shear strength) being within a preset threshold range.

[0034] During training, the detonation design parameters of each peripheral eye within the historical blasting area, along with the blasting effect of each peripheral eye within the historical blasting area, are first used as input vectors. These input vectors are then processed using a policy network to obtain the detonation design parameters of the first peripheral eye in the training area. Next, the detonation design parameters of the first peripheral eye are added to the input vector, and the policy network is used to process the new input vector to obtain the detonation design parameters of the second peripheral eye in the training area. The detonation design parameters of the second peripheral eye are then added to the input vector, and the policy network is used to process the new input vector to obtain the detonation design parameters of the third peripheral eye in the training area. This process of updating the input vector and processing it using the policy network yields the detonation design parameters of each peripheral eye within the training area.

[0035] The aforementioned policy network can be a network model with a self-attention mechanism, such as the Transformer model. The initial input vector of the policy network consists of the following: blasting design parameters for each peripheral eye within the historical blasting area (including the thickness of the blast layer, borehole spacing, charge length, decoupling coefficient, and charge structure); blasting effects for each peripheral eye within the historical blasting area (generally the blasting effects near the peripheral eye, specifically including the over-excavation volume per unit length, the under-excavation volume per unit length, and the maximum linear over-excavation length); the overall over-excavation volume; the overall under-excavation volume; and the overall maximum linear over-excavation length (i.e., the maximum over-excavation value). As each new borehole (i.e., peripheral eye) design is completed, the model's input state (i.e., input vector) is updated with the design schemes of the new boreholes (i.e., peripheral eyes) sequentially. The input state is dynamically updated as the design progresses. The generation of blasting design parameters for new peripheral eyes is terminated when the absolute value of the total borehole spacing length of the new scheme and the total borehole spacing length of the old scheme is less than 40 cm (i.e., the generation of blasting design parameters for new peripheral eyes is terminated when the absolute value of the difference between the sum of the borehole spacings of all peripheral eyes in two consecutive peripheral eye blasting schemes is less than 40 cm). The blasting design parameters for peripheral eyes output by the policy network each time can be understood as actions in deep reinforcement learning. To ensure the robustness of model training and meet the requirements of the engineering field, the actions should be selected within a specified space. Specifically, the thickness of the light blast layer should be selected from 50 cm, 55 cm, 60 cm, 65 cm, and 70 cm; the spacing between blast holes should be selected from 40 cm, 45 cm, 50 cm, 55 cm, 60 cm, 65 cm, 70 cm, and 75 cm; the charge length should be selected from 20 cm, 40 cm, 60 cm, 80 cm, 100 cm, and 120 cm; the decoupling coefficient should be selected from 1, 1.25, 1.5, and 2.0; and the charge position (i.e., charge structure) should be selected from bottom charge, middle charge, and top charge.

[0036] The overall over-excavation volume of the blasting is the total volume of the rock mass whose actual excavation outline is located outside the designed excavation outline; the overall under-excavation volume of the blasting is the total volume of the rock mass whose actual excavation outline is located inside the designed excavation outline and has not been excavated to the required depth; the maximum linear over-excavation length of the blasting is the maximum straight-line distance of the outline protruding outward from the location of the most severe over-excavation in the entire blasting area.

[0037] Understandably, after the policy network completes the generation of an overall plan (i.e., obtains the demolition design parameters for all peripheral eyes in the training area), it will calculate the reward function to adjust the parameters of the policy network.

[0038] The reward function for training the policy network is calculated based on the local predicted blasting effect output by the three-hole blasting prediction model and the overall predicted blasting effect output by the smooth blasting effect prediction model. The smooth blasting effect prediction model is used to predict the effect of the blasting scheme output by the policy network and outputs the overall predicted blasting effect; the three-hole blasting prediction model is used to predict the effect of a part of the blasting scheme output by the policy network and outputs the local predicted blasting effect.

[0039] To facilitate the calculation of the reward function, the above-mentioned method for generating peripheral eye blasting schemes also includes the following steps: during the training of the policy network, for the blasting schemes output by the policy network, according to the generation order of the blasting design parameters of the peripheral eyes, the three peripheral eyes corresponding to every three adjacent blasting design parameters are taken as a blasting unit.

[0040] Based on this, the reward function for: ; in, Indicates the number of explosive units. This represents the reduction factor. The specific value can be set according to the actual situation. Indicates explosive unit Corresponding localized explosion effect rewards, This indicates the overall explosion effect reward.

[0041] ; This indicates that the three-hole blasting prediction model is effective for blasting units. The local predicted explosion effect output during effect prediction. Indicates the effect of blasting on a local target. The specific value can be set according to the actual situation.

[0042] ; This represents the overall predicted blasting effect output by the smooth blasting effect prediction model when predicting the effect of the blasting scheme output by the policy network. Indicates the overall target blasting effect. The specific values ​​can be set according to the actual situation. A reward is given when the model's effect achieves the specified target explosion effect.

[0043] Once the model's overall calculations converge, the model training is complete. The aforementioned reward signals are ultimately transmitted to the actors. The value function in the Actor-Critic structure evaluates and guides the policy network update to ensure the stability of the model training process. It is understood that this application uses a deep reinforcement learning training framework to train the policy network, therefore the training logic will not be elaborated upon here. For example, training termination conditions could include the average cumulative reward stabilizing over multiple rounds, or the distribution of output actions from the actor network stabilizing.

[0044] The following section explains the three-hole blasting prediction model and the smooth blasting effect prediction model used to calculate the reward function during the training process.

[0045] The three-hole blasting prediction model was trained on a regression prediction network using a simulation training dataset. Each training sample in the dataset consisted of blasting data from two identical rock mass conditions. The blasting data included: blasting design parameters for a blasting group consisting of three adjacent peripheral holes, and the blasting effect of the central peripheral hole within the blasting group. The blasting design parameters for the blasting group included the blasting design parameters for each of the three adjacent peripheral holes. It should be noted that the blasting effect of the central peripheral hole in one of the blasting data points in the training samples served as supervisory information.

[0046] Specifically, a three-hole peripheral blasting model (i.e., a blasting group / blasting element model) can be constructed using ANSYS software (ANSYS is a computer-aided engineering software whose core function is to simulate real physical phenomena in a virtual environment through finite element analysis). Numerical calculations are then performed using LS-DYNA software (LS-DYNA is an explicit dynamic finite element analysis software whose core function is to perform high-precision numerical simulations of highly nonlinear, transient large deformation, and multiphysics coupling problems). The model design parameters include rock mass parameters, the thickness of the blasting layer, the spacing between the left and right boreholes, and the charge amount, decoupling coefficient, and charge structure for the left, center, and right boreholes. The model parameters consist of two parts: manually designed parameters to provide relatively standardized blasting schemes, and machine-generated parameters to expand the diversity of schemes, ensuring that the model maintains good predictive accuracy even when facing diverse schemes. After numerical calculations are completed, the over-excavation volume per unit length, the under-excavation area per unit length, and the maximum linear over-excavation length near the central borehole are collected. This represents the blasting effect of the central peripheral eye (the peripheral eye located in the middle of three adjacent peripheral eyes) of the blasting group. To reduce the prediction pressure on the network while also considering the requirements of on-site engineering construction, the borehole parameters need to be discretized: rock mass parameters should be diversified; the thickness of the smooth blasting layer should be selected from 50cm, 55cm, 60cm, 65cm, and 70cm; the borehole spacing should be selected from 40cm, 45cm, 50cm, 55cm, 60cm, 65cm, 70cm, and 75cm; the borehole charge length should be selected from 20cm, 40cm, 60cm, 80cm, 100cm, and 120cm; the decoupling coefficient should be selected from 1, 1.25, 1.5, and 2.0; and the charge structure should be selected from bottom charge, middle charge, and top charge.

[0047] Based on the numerical calculation results, a simulation training dataset is constructed. Each training sample in the simulation training dataset contains a blasting design scheme and its corresponding center borehole blasting effect. Data under the same rock mass conditions in the dataset are paired, and the resulting data contains two sets of blasting schemes and their corresponding blasting effects. For ease of description later, according to the order of combination, the data listed first is called the previous blasting scheme, and the data listed later is called the current blasting scheme. The data obtained from the pairwise combinations is used as a training sample.

[0048] In the simulation training dataset, a training sample should include the design scheme of previous numerical blasting (i.e., the blasting design parameters of the blasting group), the blasting effect calculated under the previous blasting scheme (i.e., the blasting effect of the center and peripheral holes of the blasting group), and the design scheme of the current numerical blasting (i.e., the blasting design parameters of the blasting group). Specific input parameters include the thickness of the blasting layer, the spacing between the left and right boreholes, the charge length of the three boreholes, the decoupling coefficient of the three boreholes, the charge structure of the three boreholes, the over-excavation volume per unit length of the center borehole in the previous blasting, the under-excavation area per unit length, and the maximum linear over-excavation length of the current blasting scheme; and the thickness of the blasting layer, the spacing between the left and right boreholes, the charge length of the three boreholes, the decoupling coefficient of the three boreholes, and the charge structure of the three boreholes of the current blasting scheme. The blasting effect calculated under the current blasting scheme is used as supervisory information, including the over-excavation volume per unit length, the under-excavation area per unit length, and the maximum linear over-excavation length calculated numerically for the current scheme. Meanwhile, the design schemes for two consecutive blasts in a training sample should be carried out in the same type of rock mass to reflect the similarity of rock masses in two consecutive blasts in actual engineering. The deep learning model uses the previous scheme, the effect of the previous blast, and the current scheme as input to predict the over-excavation volume per unit length, the under-excavation area per unit length, and the maximum linear over-excavation length under the current scheme. The models used include various common regression network models, such as multilayer feedforward neural networks with back propagation.

[0049] In some embodiments of this application, the smooth blasting effect prediction model is obtained by training a network model with a self-attention mechanism using an actual blasting training dataset. For example, this network model can be a Transformer model.

[0050] Each training dataset in the actual blasting training set consists of two blasting samples under the same rock mass conditions. The blasting samples include: the perimeter blasting scheme for the actual blasted area and the overall blasting effect of the actual blasted area. The actual blasted area refers to the area where perimeter blasting has actually been implemented. The perimeter blasting scheme for the actual blasted area includes the blasting design parameters for each perimeter hole within the actual blasted area. The overall blasting effect of the actual blasted area includes the overall over-excavation volume, the overall under-excavation volume, and the maximum overall over-excavation value of the actual blasted area.

[0051] Specifically, a training dataset for actual blasting can be constructed based on historically collected engineering site blasting data. A set of data (i.e., a training dataset) should include the design scheme of the previous engineering blasting, the blasting effect under the previous blasting scheme, and the design scheme of the current engineering blasting. Specifically, it includes the thickness of the blasting layer of the previous engineering blasting scheme, the spacing between each blast hole (i.e., peripheral holes), the charge length of each blast hole (i.e., peripheral holes), the decoupling coefficient of each blast hole (i.e., peripheral holes), the charge structure of each blast hole (i.e., peripheral holes), the overall over-excavation volume of the previous engineering blasting, the overall under-excavation volume of the blasting, and the maximum over-excavation value of the overall blasting, as well as the thickness of the blasting layer of the current engineering blasting scheme, the spacing between each blast hole (i.e., peripheral holes), the charge length of each blast hole (i.e., peripheral holes), the decoupling coefficient of each blast hole (i.e., peripheral holes), and the charge structure of each blast hole (i.e., peripheral holes). The blasting effect under the current blasting scheme is used as supervisory information, specifically including the overall over-excavation volume, overall under-excavation volume, and maximum overall over-excavation value of the current blasting project. The previous scheme mentioned here differs from the current scheme mentioned earlier; the two schemes should represent two consecutive blasting operations (such as blasting two adjacent construction sections of the same tunnel) to ensure the similarity of the rock mass information in the two blasts. The deep learning model should be a model based on a self-attention mechanism, such as the Transformer model.

[0052] It should be noted that the specific training methods for the three-hole blasting prediction model and the smooth blasting effect prediction model can both adopt the conventional training methods of deep learning models. Therefore, the specific training process will not be elaborated on here, and the training data has been described in detail above.

[0053] Step 2: Obtain the blasting plan and blasting effect of the blasted area; the blasted area and the target area have the same rock mass conditions.

[0054] The target area is the area where a perimeter blasting scheme needs to be designed. The blasted area has the same rock mass conditions as the target area, and perimeter blasting has already been carried out in the blasted area.

[0055] For example, the target area and the blasted area can be adjacent construction sections in the same tunnel project.

[0056] It should be noted that the blasting plan for the blasted area includes the blasting design parameters for each peripheral hole within the blasted area. This blasting plan was designed by relevant personnel based on the actual situation.

[0057] Step 3: Construct an input vector based on the blasting scheme and blasting effect of the blasted area, and process the constructed input vector using the optimal policy network to obtain the peripheral eye blasting scheme of the target area.

[0058] The aforementioned blasting scheme for the perimeter eyes of the target area includes blasting design parameters for multiple perimeter eyes of the target area. These blasting design parameters are generated one by one by the optimal strategy network. The optimal strategy network obtains the blasting design parameters for multiple perimeter eyes based on multiple distinct input vectors, with each input vector corresponding to a specific blasting design parameter for the multiple perimeter eyes.

[0059] In some embodiments of this application, step 3 above is specifically implemented by the following steps: Step 3.1: Use the blasting scheme and blasting effect of the blasted area as the input vector of the optimal strategy network.

[0060] The blasting plan for the blasted area includes the blasting design parameters for each peripheral hole in the blasted area. The blasting effect of the blasted area includes: the blasting effect of each peripheral hole in the blasted area, the overall over-excavation volume of the blasted area, the overall under-excavation volume of the blasted area, and the overall maximum over-excavation value of the blasted area.

[0061] The blasting design parameters include the thickness of the blasting layer, the spacing between boreholes, the charge length, the decoupling coefficient, and the charge structure. The blasting effect of peripheral holes generally refers to the blasting effect near the peripheral holes, specifically including the over-excavation volume per unit length, the under-excavation volume per unit length, and the maximum linear over-excavation length.

[0062] Step 3.2: Process the input vector using the optimal policy network to generate a blasting design parameter for the peripheral eye, and determine whether the blast hole generation stop condition is met. If it is met, use the generated blasting design parameter for the peripheral eye as the blasting scheme for the target area. Otherwise, proceed to step 3.3.

[0063] The blasting design parameters include the thickness of the blasting layer, the borehole spacing, the charge length, the decoupling coefficient, and the charge structure. The blasting layer thickness should be selected from 50cm, 55cm, 60cm, 65cm, and 70cm; the borehole spacing should be selected from 40cm, 45cm, 50cm, 55cm, 60cm, 65cm, 70cm, and 75cm; the borehole charge length (i.e., the aforementioned charge length) should be selected from 20cm, 40cm, 60cm, 80cm, 100cm, and 120cm; the decoupling coefficient should be selected from 1, 1.25, 1.5, and 2.0; and the charge structure should be selected from bottom charge, middle charge, and top charge.

[0064] The above-mentioned condition for stopping borehole generation is: the absolute value of the difference between the sum of the borehole spacing of all peripheral holes in the current blasting scheme and the sum of the borehole spacing of all peripheral holes in the previous blasting scheme is less than a preset value (the specific data of the preset value can be set to 40 cm, as in the training process). The current blasting scheme includes: the blasting design parameters of all peripheral holes generated up to the current moment; the previous blasting scheme includes: the blasting design parameters of all peripheral holes in the current blasting scheme except for the blasting design parameters of the last generated peripheral hole.

[0065] Step 3.3: Add the generated blasting design parameters of the peripheral eye to the input vector, use the input vector with the generated blasting design parameters of the peripheral eye as the input vector in step 3.2, and return to execute step 3.2.

[0066] Overall, this application combines supervised learning with the Actor-Critic algorithm in deep reinforcement learning to construct a reward function for peripheral eye blasting. By inferring the physical properties of the current rock mass from the actual effect of the previous blast, it achieves refined design and automated construction scheme generation for each peripheral eye. This solves the problem of using the same scheme indiscriminately from beginning to end in existing blasting projects, significantly improving the accuracy and efficiency of blasting construction, and providing a feasible intelligent technical solution for the refined blasting concept of "one blast, one design".

[0067] Specifically, the peripheral eye explosion scheme generation method based on deep reinforcement learning provided in this application has the following advantages: I. The peripheral hole blasting scheme designed in this application can be designed hole by hole according to the local rock strata characteristics, avoiding the problem of using uniform parameters and ignoring the differences in local rock strata in the design of existing blasting schemes. It effectively prevents over-excavation and under-excavation when there are weak interlayers or hard rocks in the rock strata, and provides an effective technical means for the scientific optimization of blasting schemes.

[0068] Second, this application applies deep reinforcement learning methods to the design of perimeter eyes in blasting, which can effectively improve the accuracy and adaptability of the design scheme. At the same time, it uses numerical simulation methods to supplement the insufficient amount of existing data, avoiding the training risks and safety hazards caused by directly using the final blasting effect as a reward. It overcomes the technical bottlenecks of traditional reinforcement learning methods in blasting engineering applications, such as difficulty in reward design and unstable training.

[0069] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for generating peripheral eye explosion schemes based on deep reinforcement learning, characterized in that, include: Step 1: Train the policy network using deep reinforcement learning to obtain the optimal policy network. The reward function during policy network training is calculated based on the local predicted blasting effect output by the three-hole blasting prediction model and the overall predicted blasting effect output by the smooth blasting effect prediction model. The smooth blasting effect prediction model is used to predict the effect of the blasting scheme output by the policy network and output the overall predicted blasting effect. The three-hole blasting prediction model is used to predict the effect of a part of the blasting scheme output by the policy network and output the local predicted blasting effect. Step 2: Obtain the blasting plan and blasting effect of the blasted area; the blasted area and the target area have the same rock mass conditions; Step 3: Construct an input vector based on the blasting scheme and blasting effect of the blasted area, and process the constructed input vector using the optimal strategy network to obtain the peripheral eye blasting scheme of the target area; the peripheral eye blasting scheme of the target area includes blasting design parameters of multiple peripheral eyes of the target area, and the blasting design parameters of multiple peripheral eyes are generated one by one by the optimal strategy network. The optimal strategy network obtains multiple blasting design parameters of multiple peripheral eyes based on multiple distinct input vectors, and the multiple input vectors correspond one-to-one with the multiple blasting design parameters of multiple peripheral eyes.

2. The method for generating a peripheral eye blasting scheme according to claim 1, characterized in that, The process of constructing an input vector based on the blasting scheme and blasting effect of the blasted area, and processing the constructed input vector using the optimal policy network to obtain the peripheral eye blasting scheme of the target area, includes: Step 3.1: Use the blasting scheme and blasting effect of the blasted area as the input vector of the optimal strategy network; Step 3.2: Process the input vector using the optimal strategy network to generate a blasting design parameter for the peripheral eye, and determine whether the blast hole generation stop condition is met. If it is met, use the generated blasting design parameter for the peripheral eye as the blasting scheme for the peripheral eye of the target area; otherwise, proceed to step 3.

3. Step 3.3: Add the generated blasting design parameters of the peripheral eye to the input vector, use the input vector with the generated blasting design parameters of the peripheral eye as the input vector in step 3.2, and return to execute step 3.

2.

3. The method for generating a peripheral eye blasting scheme according to claim 2, characterized in that, The blasting scheme of the blasted area includes the blasting design parameters of each peripheral hole in the blasted area, and the blasting effect of the blasted area includes: the blasting effect of each peripheral hole in the blasted area, the overall over-excavation volume of the blasted area, the overall under-excavation volume of the blasted area, and the overall maximum over-excavation value of the blasting.

4. The method for generating a peripheral eye blasting scheme according to claim 3, characterized in that, Blasting design parameters include the thickness of the blasting layer, the spacing between boreholes, the charge length, the decoupling coefficient, and the charge structure.

5. The method for generating a peripheral eye blasting scheme according to claim 4, characterized in that, The condition for stopping borehole generation is: the absolute value of the difference between the sum of the borehole spacing of all peripheral holes in the current blasting scheme and the sum of the borehole spacing of all peripheral holes in the previous blasting scheme is less than a preset value. The current blasting scheme includes: the blasting design parameters of all peripheral holes generated up to the current moment; the previous blasting scheme includes: the blasting design parameters of all peripheral holes in the current blasting scheme except for the blasting design parameters of the last generated peripheral hole.

6. The method for generating a peripheral eye blasting scheme according to claim 1, characterized in that, The three-hole blasting prediction model is obtained by training a regression prediction network using a simulation training dataset. Each training sample in the simulation training dataset consists of blasting data under two identical rock mass conditions. The blasting data includes: blasting design parameters of a blasting group consisting of three adjacent peripheral holes, and the blasting effect of the central peripheral hole of the blasting group.

7. The method for generating a peripheral eye blasting scheme according to claim 1, characterized in that, The smooth blasting effect prediction model is obtained by training a network model with a self-attention mechanism using an actual blasting training dataset. Each training data point in the actual blasting training dataset consists of two blasting samples under the same rock mass conditions. The blasting samples include: the perimeter hole blasting scheme of the actual blasted area and the overall blasting effect of the actual blasted area.

8. The method for generating a peripheral eye blasting scheme according to claim 1, characterized in that, The reinforcement learning training dataset used to train the policy network includes: the blasting design parameters of each peripheral eye within the historical blasting area, and the blasting effect of each peripheral eye within the historical blasting area.

9. The method for generating a peripheral eye blasting scheme according to claim 8, characterized in that, During the training of the policy network, the blasting scheme output by the policy network includes: blasting design parameters for multiple peripheral holes in a training area with the same rock mass conditions as historical blasting areas; the blasting design parameters for multiple peripheral holes in the training area are generated one by one; the peripheral hole blasting scheme generation method further includes: During the training of the policy network, for the blasting scheme output by the policy network, according to the generation order of the blasting design parameters of the peripheral eyes, the three peripheral eyes corresponding to every three adjacent blasting design parameters are taken as a blasting unit.

10. The method for generating a peripheral eye blasting scheme according to claim 9, characterized in that, reward function for: ； in, Indicates the number of explosive units. This represents the reduction factor. Indicates explosive unit Corresponding localized explosion effect rewards, This indicates the overall explosion effect reward; ； This indicates that the three-hole blasting prediction model is effective for blasting units. The local predicted explosion effect output during effect prediction. Indicates the effect of blasting on a local target; ； This represents the overall predicted blasting effect output by the smooth blasting effect prediction model when predicting the effect of the blasting scheme output by the policy network. This indicates the overall effect of the explosion on the target.

Images