An AI-based milling excavation boundary control method and system

Abstract

The present application provides an AI-based milling excavation boundary control method and system. The first image acquisition device disposed on the milling excavation equipment performs image acquisition in the direction away from the heading face of the target tunnel to obtain first image data, and the pre-trained correction prediction model determines the first distortion correction parameter according to at least three target positioning markers in the first image data. Then, the projection device disposed on the milling excavation equipment projects the structured light pattern corrected by the first distortion correction parameter towards the heading face. In this way, even when the milling excavation equipment moves and operates in the target tunnel, it can automatically project a suitable structured light pattern indicating the milling excavation boundary on the heading face, and the construction personnel can more clearly determine the milling excavation boundary according to the structured light pattern, thereby reducing the risk of under-excavation or over-excavation during the milling excavation process.

Description

Technical Field

[0001] This application relates to the field of AI image processing, and more specifically, to an AI-based milling excavation boundary control method and system. Background Art

[0002] In some tunnel construction operations, milling excavation equipment is used for tunnel excavation to reduce the impact of excavation operations on the surrounding rock strata or surface facilities. During the milling excavation operation, the milling excavation equipment performs milling excavation operations on the heading face through a milling head. After milling a certain depth, support is carried out, and then further milling excavation is carried out, with the support and milling excavation operations alternating. In the existing milling excavation scheme, construction workers operate a boom with a milling head on the milling excavation equipment to perform milling excavation operations on the heading face. Usually, the telescopic direction of the boom is consistent with the extension direction of the construction worker's line of sight, which poses a certain difficulty for the construction worker to judge the extension distance of the boom. When extending the boom from the middle of the tunnel to mill the edge of the heading face, it is easy to cause under-excavation or over-excavation due to inaccurate judgment of the milling depth, and subsequent re-construction is required for remedy, affecting the construction efficiency. Summary of the Invention

[0003] In order to overcome the above deficiencies in the prior art, the purpose of this application is to provide an AI-based milling excavation boundary control method, and the method includes:

[0004] Performing image acquisition in the direction away from the heading face of the target tunnel by a first image acquisition device disposed on the milling excavation equipment to obtain first image data including at least three target positioning markers in the excavated area of the target tunnel;

[0005] Inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter;

[0006] Projecting a structured light pattern for indicating the milling excavation boundary towards the heading face by a projection device disposed on the milling excavation equipment, and performing slope distortion correction on the structured light pattern according to the first distortion correction parameter;

[0007] Performing milling excavation operations on the heading face according to the indication of the structured light pattern.

[0008] In some possible implementation manners, the step of inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter includes:

[0009] Inputting the first image data into the feature extraction module of the correction prediction model, and identifying the positions of at least three of the target positioning markers in the first image data through the feature extraction module to obtain a position feature vector;

[0010] Input the position feature vector into the regression prediction module of the correction prediction model to obtain the first distortion correction parameter determined by the regression prediction module according to the position feature vector.

[0011] In some possible implementation manners, the method further includes:

[0012] Obtain the device parameters of the milling excavation device, where the milling excavation device includes a main body and a cantilever provided with a milling head, and the device parameters include an operation distance parameter characterizing the distance range between the main body of the milling excavation device and the heading face during the milling excavation operation;

[0013] The step of inputting the first image data into a pre-trained correction prediction model for processing to obtain the first distortion correction parameter includes:

[0014] Input the first image data and the device parameters into a pre-trained correction prediction model for processing to obtain the first distortion correction parameter.

[0015] In some possible implementation manners, the step of projecting a structured light pattern for indicating the milling boundary towards the heading face by a projection device disposed on the milling excavation device includes:

[0016] Obtain the tunnel excavation parameters of the target tunnel, where the tunnel excavation parameters are used to indicate the size and / or shape that needs to be excavated on the inner wall of the tunnel;

[0017] Generate a corresponding structured light pattern according to the tunnel excavation parameters, and project the structured light pattern towards the heading face by a projection device disposed on the milling excavation device.

[0018] In some possible implementation manners, the method further includes:

[0019] Obtain the working state data of the milling excavation device;

[0020] According to the working state data, when it is detected that the milling excavation device drives the first image acquisition device and / or the projection device to move or rotate in position, re-acquire new first image data and project a new structured light pattern.

[0021] In some possible implementation manners, the structured light pattern includes a grid pattern; the method further includes:

[0022] Perform image acquisition towards the heading face by a second image acquisition device disposed on the milling excavation device to obtain second image data including the structured light pattern;

[0023] Determine a position correction parameter according to the distortion of the structured light pattern at the junction of the heading face and the tunnel sidewall in the second image data.

[0024] Adjust the projection position of the structured light pattern according to the position correction parameter.

[0025] In some possible implementation manners, the method further includes:

[0026] Adjust the projection colors at different projection positions on the structured light pattern according to the positions of the concavo-convex distortions of the structured light pattern in the second image data.

[0027] In some possible implementation manners, the method further includes:

[0028] Input the second image data into a pre-trained content recognition model, and determine, through the content recognition model, a first area formed by the milled slag accumulation and a second area on the heading face that is not covered by the milled slag accumulation in the second image data;

[0029] The step of adjusting the projection colors at different projection positions on the structured light pattern according to the positions of the concavo-convex distortions of the structured light pattern in the second image data includes:

[0030] For the second area, adjust the projection colors at different projection positions on the structured light pattern according to the positions of the concavo-convex distortions of the structured light pattern in the second image data.

[0031] In some possible implementation manners, the method further includes:

[0032] Perform image acquisition in the direction away from the heading face of the sample tunnel through the first image acquisition device disposed on the milling equipment, to obtain first sample image data including at least three target positioning markers in the excavated area of the target tunnel;

[0033] Project a sample structured light pattern towards the heading face through the projection device disposed on the milling equipment, and adjust the slope distortion correction parameter of the sample structured light pattern, to obtain a first sample correction parameter;

[0034] Use the plurality of first sample images and the corresponding first sample correction parameters to train the correction prediction model to be trained, so that the correction prediction model outputs corresponding distortion correction parameters according to the input image data.

[0035] Another object of the present application is to provide an AI-based milling boundary control system, the system includes a data processing device, a first image acquisition device and a projection device; the first image acquisition device and the projection device are disposed on the milling equipment;

[0036] The first image acquisition device is used to perform image acquisition in the direction away from the heading face of the target tunnel, and obtain first image data including at least three target positioning markers in the excavated area of the target tunnel;

[0037] The data processing device is used to input the first image data into a pre-trained correction prediction model for processing to obtain first distortion correction parameters;

[0038] The data processing device is further used to project a structured light pattern for indicating the milling boundary towards the heading face through the milling equipment, and perform slope distortion correction on the structured light pattern according to the first distortion correction parameters, so that the construction personnel and / or the milling equipment perform milling operations on the heading face according to the indication of the structured light pattern.

[0039] Compared with the prior art, the present application has the following beneficial effects:

[0040] The AI-based milling boundary control method and system provided by the present application perform image acquisition in the direction away from the heading face of the target tunnel through the first image acquisition device provided on the milling equipment to obtain first image data, and determine first distortion correction parameters according to at least three target positioning markers in the first image data through a pre-trained correction prediction model, and then project a structured light pattern corrected by the first distortion correction parameters towards the heading face through the projection device provided on the milling equipment. In this way, when the milling equipment moves and operates in the target tunnel, a suitable structured light pattern for indicating the milling boundary can also be automatically projected on the heading face, and the construction personnel can more clearly determine the milling boundary according to the structured light pattern, thereby reducing the risk of under-excavation or over-excavation during the milling process. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1 It is a schematic diagram of the AI-based milling boundary control system provided by the embodiment of the present application;

[0043] Figure 2 It is one of the application scenario schematic diagrams of the AI-based milling boundary control system provided by the embodiment of the present application;

[0044] Figure 3 It is a schematic diagram of the step flow of the AI-based milling boundary control method provided by the embodiment of the present application;

[0045] Figure 4 This is the second schematic diagram of the application scenario of the AI-based milling boundary control system provided by the embodiments of the present application;

[0046] Figure 5 This is the third schematic diagram of the application scenario of the AI-based milling boundary control system provided by the embodiments of the present application;

[0047] Figure 6 This is the schematic diagram of the data device provided by the embodiments of the present application. Detailed implementation manners

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Usually, the components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0049] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application to be protected, but merely represents the selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0050] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0051] In the description of the present application, it should be noted that the terms "first", "second", "third", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0052] In the description of the present application, it should also be noted that unless otherwise clearly defined and limited, the terms "set", "installed", "connected", and "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific situations.

[0053] Please refer to Figure 1 , Figure 1A schematic diagram of an AI-based milling and excavation boundary control system provided in this embodiment, the system may include a data processing device 100, a first image acquisition device 200 and a projection device 300.

[0054] See also Figure 2 , Figure 2 The schematic diagram of the application scenario of the solution provided in this embodiment is that the milling and digging equipment 600 may be a cantilever milling and digging equipment 600 with a milling and digging head, and the milling and digging equipment 600 may include a body and a cantilever. The first image acquisition device 200 and the projection device 300 may be arranged on the milling and digging equipment 600. The first image acquisition device 200 may be arranged on the body, and the acquisition direction of the first image acquisition device 200 may deviate from the extension direction of the cantilever, and the projection device 300 may be arranged on the body, and the projection direction of the projection device 300 may be along the extension direction of the cantilever.

[0055] For example, the body of the milling and digging equipment 600 is provided with a cab, and during the operation of the milling and digging equipment 600, the driver's seat in the cab is usually facing the tunnel face 700. The projection direction of the projection device 300 can be consistent with the direction of the driver's seat, and the image acquisition direction of the first image acquisition device 200 can be opposite to the direction of the driver's seat.

[0056] The data processing device 100, the first image acquisition device 200 and the projection device 300 cooperate with each other to implement the AI-based milling and excavation boundary control method provided in this embodiment.

[0057] Specifically, see Figure 3 The AI-based milling and excavation boundary control method may include the following steps.

[0058] Step S110, performing image acquisition in a direction away from the tunnel face 700 of the target tunnel by a first image acquisition device arranged on the milling and excavation device 600, and obtaining first image data of at least three target positioning markers 900 including the excavated area of ​​the target tunnel.

[0059] See also Figure 4 In this embodiment, the target tunnel may include an excavated area and an area to be excavated, wherein the excavated area may be an area where excavation or support has been completed, and the inner wall of the tunnel in this area will not be subjected to a large number of construction operations in a short period of time. The target positioning marker 900 may be respectively set at at least three different specific locations in the area. The target positioning marker 900 may be an indicator light of a special color. For example, the target positioning marker 900 may be respectively set at the arch, the middle of the tunnel wall on both sides, and the bottom of the tunnel side walls on both sides.

[0060] In this embodiment, when the milling and excavation equipment 600 performs milling and excavation operations towards the tunnel face 700, the first image acquisition device 200 disposed on the milling and excavation equipment 600 can face the excavated area to perform image acquisition, and the acquired images can include at least three of the above-mentioned target positioning markers 900.

[0061] It should be noted that as the excavation of the tunnel face 700 progresses, the positions of the target positioning markers 900 can also be adjusted accordingly, but the advancement speed of the target positioning markers 900 can be much slower than the advancement speed of the tunnel face 700. For example, the milling and excavation equipment 600 alternately performs milling and support at a speed of advancing 1 meter per excavation, and after milling and advancing 50 times (i.e., advancing 50 meters), the target positioning markers 900 are reinstalled or set 50 meters along the tunnel extension direction. In this way, it can be ensured that the images collected by the first image acquisition device 200 can clearly capture the target positioning markers 900, and it can also avoid the frequent movement of the target positioning markers 900 from affecting the construction progress.

[0062] It should be noted that in this embodiment, the installation positions of the respective target positioning markers 900 should be relatively fixed so as to determine the relative position relationship between the milling and excavation equipment 600 and each target positioning marker 900 in subsequent steps.

[0063] Step S120: Input the first image data into a pre-trained correction and prediction model for processing to obtain the first distortion correction parameters.

[0064] Step S130: The projection device 300 disposed on the milling and excavation equipment 600 projects a structured light pattern for indicating the milling boundary towards the tunnel face 700, and corrects the slope distortion of the structured light pattern according to the first distortion correction parameters.

[0065] Step S140: Perform milling and excavation operations on the tunnel face 700 according to the indication of the structured light pattern.

[0066] In this embodiment, in step S130, when the milling and excavation equipment 600 performs milling and excavation operations towards the tunnel face 700, the projection device 300 disposed on the milling and excavation equipment 600 can project a structured light pattern on the tunnel face 700. The structured light pattern can include a stripe pattern or a grid pattern.

[0067] When the structured light pattern is projected onto a relatively flat plane, the stripes or grids therein will not be distorted, but when projected onto an uneven plane or at the junction of planes, the stripes and grids therein will be distorted accordingly.

[0068] In this embodiment, when the structured light pattern is projected at the junction of the tunnel face 700 and the tunnel side wall, the distortion of the structured light pattern can clearly show the current junction position of the tunnel face 700 and the side wall. In addition, different shaped lines or lines of different colors can also be projected in the structured light pattern to indicate the desired milling boundary indication line. In this way, in step S140, the construction worker operating the milling equipment 600 can see the desired milling boundary indication line from the structured light pattern and clearly see the current milling boundary according to the distortion of the structured light pattern, so that the milling operation can be better performed accordingly.

[0069] In this embodiment, please refer to Figure 5 , during the process of the milling equipment 600 performing the milling operation, the position of the milling equipment 600 relative to the tunnel face 700 may move, and the milling equipment 600 does not always perform the milling operation perpendicular to the tunnel face 700. When the projection device 300 projects the structured light pattern towards the tunnel face 700, the projection angle and the projection direction will also change accordingly. Different projection angles and projection directions will cause the projected structured light pattern to produce slope distortion.

[0070] Therefore, step S120 can be performed before the projection. In step S120, the collected first image data can be input into a pre-trained correction prediction model for processing, and the correction prediction model can determine the corresponding first distortion correction parameter according to the first image data.

[0071] Specifically, step S120 can include the following sub-steps.

[0072] Step S121, input the first image data into the feature extraction module of the correction prediction model, and identify the positions of at least three of the target positioning markers 900 in the first image data through the feature extraction module to obtain a position feature vector.

[0073] That is, the feature extraction module can perform image content recognition on the first image data and determine the positions of at least three of the target positioning markers 900 in the first image data as the position feature vector.

[0074] Specifically, when the milling and excavation device 600 is in different positions, the shooting angle or shooting direction of the first image acquisition device 200 is different. In this case, the relative positions of the target positioning markers 900 captured in the first image data are also different. Therefore, the positions of the respective target positioning markers 900 in the first image data can also characterize the relative position relationship between the milling and excavation device 600 and the respective target positioning markers 900. Since the relative positions of the respective target positioning markers 900 and the heading face 700 are fixed, the position feature vectors obtained in step S121 can also characterize the relative position relationship between the milling and excavation device 600 and the heading face 700.

[0075] Step S122: Input the position feature vector into the regression prediction module of the correction prediction model to obtain the first distortion correction parameter determined by the regression prediction module according to the position feature vector.

[0076] Specifically, after determining the position feature vector in step S121, regression prediction can be performed according to the position feature vector in step S122 to determine the corresponding first distortion correction parameter. The first distortion correction parameter can be used to correct the slope distortion of the structured light image in step S130, so as to offset the distortion generated by the projection device 300 during the projection process due to the inclined projection towards the heading face 700.

[0077] Based on the above design, in the solution provided in this embodiment, the first image acquisition device 200 disposed on the milling and excavation device 600 performs image acquisition in the direction away from the heading face 700 of the target tunnel to obtain the first image data, and the first distortion correction parameter is determined by the pre-trained correction prediction model according to at least three target positioning markers 900 in the first image data. Then, the structured light pattern corrected by the first distortion correction parameter is projected towards the heading face 700 by the projection device 300 disposed on the milling and excavation device 600. In this way, when the milling and excavation device 600 moves and operates in the target tunnel, a suitable structured light pattern indicating the milling boundary can be automatically projected on the heading face 700, and the construction personnel can more clearly determine the milling boundary according to the structured light pattern, thereby reducing the risk of under-excavation or over-excavation during the milling process.

[0078] In some possible implementation manners, the method provided in this embodiment may further include step S210.

[0079] Step S210: Obtain the device parameters of the milling and excavation device 600. The milling and excavation device 600 includes a main body and a boom provided with a milling head. The device parameters include an operation distance parameter characterizing the distance range between the main body of the milling and excavation device 600 and the heading face 700 during the milling operation.

[0080] Specifically, during the milling operation with the rotation of the milling head of the same milling equipment 600, the main body of the milling equipment 600 usually does not move. The telescopic movement of the boom drives the milling head to perform milling at different positions. Therefore, during this process, the projected structured light pattern does not need to be adjusted.

[0081] However, among different milling equipment 600, the boom lengths are different. Therefore, the distance between the main body of the milling equipment 600 and the tunnel face 700 is also different, which in turn leads to different projection distances of the projection device 300. To avoid image distortion or change in the projection image range caused by different projection distances, in step S130, the first image data and the equipment parameters can be input into a pre-trained correction prediction model for processing to obtain the first distortion correction parameters. That is, the first distortion correction parameters are determined based on the first image data and the equipment parameters together.

[0082] In some possible implementation manners, step S130 may further include the following sub-steps.

[0083] Step S131, obtain the tunnel excavation parameters of the target tunnel, where the tunnel excavation parameters are used to indicate the size and / or shape that needs to be excavated on the inner wall of the tunnel.

[0084] In this embodiment, the design requirements of the target tunnel can be obtained in advance, and the tunnel excavation parameters can be input according to the design requirements.

[0085] Step S132, generate a corresponding structured light pattern according to the tunnel excavation parameters, and project the structured light pattern onto the tunnel face 700 through the projection device 300 disposed on the milling equipment 600.

[0086] In this embodiment, a corresponding structured light pattern can be generated according to the tunnel excavation parameters. For example, in a structured light pattern in the form of stripes or grids, milling boundary indication lines that conform to the tunnel excavation parameters are projected through different lines or different colors.

[0087] It should be noted that in this embodiment, in order to better display the current milling boundary through the distortion of the structured light pattern, the coverage area of the structured light pattern needs to be larger than the area of the tunnel face 700, and at least part of the structured light pattern needs to cover the junction of the tunnel face 700 and the tunnel side wall, or cover the junction of the tunnel face 700 and the ground.

[0088] In some possible implementation manners, the working state data of the milling excavation device 600 may be acquired first, and then, according to the working state data, when it is detected that the milling excavation device 600 drives the first image acquisition device 200 and / or the projection device 300 to move or rotate in position, new first image data is re-acquired and a new structured light pattern is projected.

[0089] Specifically, during the process of the milling head of the milling excavation device 600 rotating to perform milling excavation, the main body of the milling excavation device 600 generally does not move or rotate, and only the cantilever moves to drive the milling head to perform milling excavation. Therefore, during this stage, a new slope distortion correction does not need to be performed on the projected structured light pattern.

[0090] After a small area of milling excavation is completed, the main body of the milling excavation device 600 moves, or rotates and moves by an angle to perform milling excavation on another small area. In this case, the projection device 300 moves or rotates along with the main body of the milling excavation device 600. Therefore, whenever it is detected that the milling excavation device 600 drives the first image acquisition device 200 and / or the projection device 300 to move or rotate in position, new first image data is re-acquired and a new structured light pattern is projected, that is, a new projection and slope distortion correction action of the structured light image is re-performed to ensure the accuracy of the projected structured light image.

[0091] In some possible implementation manners, the structured light pattern includes a grid pattern. The solution provided in this embodiment may further include the following steps.

[0092] Step S310: The second image acquisition device disposed on the milling excavation device 600 performs image acquisition towards the heading face 700 to obtain second image data including the structured light pattern.

[0093] Step S320: Determine position correction parameters according to the distortion generated by the structured light pattern at the junction of the heading face 700 and the tunnel sidewall in the second image data.

[0094] Step S330: Adjust the projection position of the structured light pattern according to the position correction parameters.

[0095] Specifically, in this embodiment, in step S120, the correction prediction model may determine a position feature vector that can roughly represent the relative position relationship between the milling excavation device 600 and the heading face 700 according to the positions of the respective target positioning markers 900 in the first image data. However, for different milling excavation devices 600, the installation position of the projection device 300 or the overall height of the milling excavation device 600 may be different.

[0096] Moreover, since the structured light pattern is distorted at the boundary of the current heading face 700, in step S310, the second image acquisition device can be used to acquire the second image data of the heading face 700 after the structured light pattern is projected onto the heading face 700.

[0097] In step S320 and step S330, the position of the projected structured light pattern can be adjusted according to the distortion position of the structured light pattern at the boundary of the current heading face 700 in the second image data. For example, the projected structured light pattern can be moved or scaled so that the milling boundary indication line in the structured light pattern is aligned with the position where distortion occurs at the boundary of the current heading face 700. In this way, it is ensured that the pattern projected by the projection device 300 can accurately align with the heading face 700.

[0098] In some possible implementation manners, the method provided in this embodiment may further include step S340.

[0099] Step S340: Adjust the projection color at different projection positions on the structured light pattern according to the position of the concave-convex distortion of the structured light pattern in the second image data.

[0100] Specifically, during the milling process, the area that has been milled and the area that has not been milled are uneven. To better display the area that has not been milled, when projecting the grid of the structured light pattern, the color of the lines of the grid projected to this position can be adjusted according to the position where concave-convex distortion occurs in the second image, so as to indicate the situation of the corresponding area.

[0101] For example, the grid lines at the positions protruding relative to other areas are projected in red, and the grid lines in other flat areas (areas where the grid has no distortion) are projected in green.

[0102] In some possible implementation manners, please refer back to Figure 2 , during the milling process, the rock layer broken by the milling head will fall to the bottom of the heading face 700 to form a milling slag accumulation 800. To avoid the influence of the milling slag accumulation 800 on identifying the position of concave-convex distortion in the second image, in this embodiment, the second image data can be input into a pre-trained content recognition model first, and the first area formed by the milling slag accumulation 800 and the second area of the heading face 700 that is not covered by the milling slag accumulation 800 in the second image data can be determined through the content recognition model.

[0103] Then, for the second area, the projection color at different projection positions on the structured light pattern is adjusted according to the position of the concave-convex distortion of the structured light pattern in the second image data.

[0104] In some possible implementation manners, the present embodiment further provides a training step for the correction prediction model. Specifically, the following steps may be included.

[0105] Step S510: The first image acquisition device 200 disposed on the milling excavation device 600 performs image acquisition in the direction away from the heading face 700 of the sample tunnel, and obtains first sample image data including at least three target positioning markers 900 in the excavated area of the target tunnel.

[0106] Step S520: The projection device 300 disposed on the milling excavation device 600 projects a sample structured light pattern toward the heading face 700, and adjusts the inclination distortion correction parameters of the sample structured light pattern to obtain first sample correction parameters.

[0107] Step S530: Use the plurality of first sample images and the corresponding first sample correction parameters to train the correction prediction model to be trained, so that the correction prediction model outputs corresponding distortion correction parameters according to the input image data.

[0108] Specifically, in step S510 and step S520, when the milling excavation device 600 is in the working position, the first image acquisition device 200 may be used to acquire the first sample image data. At the same time, the projection device 300 projects the sample structured light pattern, and then manually adjusts the distortion correction parameters of the projection device 300 so that there is no correction distortion in the projection of the sample structured light pattern on the heading face 700, and the currently used distortion correction parameters are used as the first sample correction parameters.

[0109] Next, in step S530, use the plurality of first sample images and the corresponding first sample correction parameters to train the correction prediction model to be trained. Among them, the correction prediction model may output second sample correction parameters according to the input first sample image data, and then adjust the network parameters of the correction prediction model according to the first sample correction parameters and the second sample correction parameters to reduce the gap between the first sample correction parameters and the second sample correction parameters.

[0110] After the number of times of training with different samples reaches a preset threshold, or the gap between the first sample correction parameters and the second sample correction parameters is less than a set threshold, the trained correction prediction model is obtained.

[0111] Please refer to Figure 6 , Figure 6 Yes Figure 1 is a block diagram of the data processing device 100 shown. The data processing device 100 includes an AI-based milling excavation boundary control device 110, a machine-readable storage medium 120, and a processor 130.

[0112] The components of the machine-readable storage medium 120 and the processor 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The AI-based milling and digging boundary control device 110 includes at least one software function module that can be stored in the form of software or firmware in the machine-readable storage medium 120 or solidified in the operating system (OS) of the data processing device 100. The processor 130 is used to execute the executable modules stored in the machine-readable storage medium 120, such as the software function modules and computer programs included in the AI-based milling and digging boundary control device 110.

[0113] The machine-readable storage medium 120 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The machine-readable storage medium 120 is used to store a program, and the processor 130 executes the program / executable AI-based milling and excavation boundary control method provided in this embodiment after receiving an execution instruction.

[0114] The processor 130 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0115] In summary, for the AI-based milling excavation boundary control method and system provided in this application, the first image acquisition device disposed on the milling excavation equipment performs image acquisition in the direction away from the heading face of the target tunnel to obtain first image data, and the pre-trained correction prediction model determines the first distortion correction parameter according to at least three target positioning markers in the first image data. Then, the projection device 300 disposed on the milling excavation equipment projects the structured light pattern corrected by the first distortion correction parameter towards the heading face. In this way, even when the milling excavation equipment is moving and operating in the target tunnel, it is also possible to automatically project an appropriate structured light pattern indicating the milling excavation boundary on the heading face, and the construction personnel can more clearly determine the milling excavation boundary based on the structured light pattern, thereby reducing the risk of under-excavation or over-excavation during the milling excavation process.

[0116] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0117] In addition, in each embodiment of this application, the various functional modules may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0118] If the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0119] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0120] As described above, these are only various implementation manners of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A milling and excavation boundary control method based on AI, characterized in that: The method comprises: By using a first image acquisition device disposed on the milling and excavation device, image acquisition is performed in a direction of the target tunnel away from the tunnel face to obtain first image data of at least three target positioning markers including an excavated area of ​​the target tunnel; Inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter; Projecting a structured light pattern for indicating a milling boundary toward the tunnel face by means of a projection device disposed on the milling device, and performing slope distortion correction on the structured light pattern according to the first distortion correction parameter; performing milling and excavation operations on the tunnel face according to the structured light pattern instructions; The step of inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter includes: Inputting the first image data into a feature extraction module of the correction prediction model, identifying the positions of at least three target positioning markers in the first image data through the feature extraction module, and obtaining a position feature vector; The position feature vector is input into a regression prediction module of the correction prediction model to obtain the first distortion correction parameter determined by the regression prediction module according to the position feature vector.

2. The method according to claim 1, characterized in that: The method further comprises: Acquire equipment parameters of the milling and digging equipment, wherein the milling and digging equipment includes a main body and a cantilever provided with a milling and digging head, and the equipment parameters include an operation distance parameter characterizing a distance range between the main body of the milling and digging equipment and a tunnel face when performing a milling and digging operation; The step of inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter includes: The first image data and the device parameters are input into a pre-trained correction prediction model for processing to obtain first distortion correction parameters.

3. The method according to claim 1, characterized in that The step of projecting a structured light pattern for indicating a milling boundary toward the tunnel face by means of a projection device disposed on the milling device comprises: Acquire tunnel excavation parameters of the target tunnel, where the tunnel excavation parameters are used to indicate the size and / or shape of the inner wall of the tunnel that needs to be excavated; A corresponding structured light pattern is generated according to the tunnel excavation parameters, and the structured light pattern is projected toward the tunnel face by a projection device arranged on the milling device.

4. The method according to claim 1, characterized in that: The method further comprises: Acquiring working status data of the milling and digging equipment; According to the working status data, when it is detected that the milling device drives the first image acquisition device and / or the projection device to move or rotate, new first image data is re-acquired and a new structured light pattern is projected.

5. The method according to claim 1, characterized in that The structured light pattern comprises a grid pattern; and the method further comprises: By using a second image acquisition device disposed on the milling device to perform image acquisition toward the tunnel face, second image data including the structured light pattern is obtained; determining a position correction parameter according to the distortion of the structured light pattern at the junction of the tunnel face and the tunnel side wall in the second image data; The projection position of the structured light pattern is adjusted according to the position correction parameter.

6. The method according to claim 5, characterized in that The method further comprises: The projection colors of different projection positions on the structured light pattern are adjusted according to the positions of the concave-convex distortion of the structured light pattern in the second image data.

7. The method according to claim 6, characterized in that The method further comprises: Inputting the second image data into a pre-trained content recognition model, and determining, by means of the content recognition model, a first area in the second image data where a milling slag deposit is formed and a second area on the tunnel face where the milling slag deposit is not covered; The step of adjusting the projection colors of different projection positions on the structured light pattern according to the position of the concave-convex distortion of the structured light pattern in the second image data comprises: For the second area, the projection colors of different projection positions on the structured light pattern are adjusted according to the positions of the concave-convex distortion of the structured light pattern in the second image data.

8. The method according to claim 1, characterized in that The method further comprises: The first image acquisition device disposed on the milling and excavation device performs image acquisition in a direction away from the tunnel face of the sample tunnel to obtain first sample image data of at least three target positioning markers including the excavated area of ​​the target tunnel; Projecting a sample structured light pattern toward the tunnel face by the projection device disposed on the milling device, and adjusting a slope distortion correction parameter of the sample structured light pattern to obtain a first sample correction parameter; The correction prediction model to be trained is trained using a plurality of the first sample images and the corresponding first sample correction parameters, so that the correction prediction model outputs corresponding distortion correction parameters according to the input image data.

9. An AI-based milling and excavation boundary control system, characterized in that: The system comprises a data processing device, a first image acquisition device and a projection device; the first image acquisition device and the projection device are arranged on the milling and digging device; The first image acquisition device is used to perform image acquisition in a direction away from the tunnel face of the target tunnel, and obtain first image data of at least three target positioning markers including the excavated area of ​​the target tunnel; The data processing device is used to input the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter; The data processing device is further used to project a structured light pattern for indicating a milling boundary toward the tunnel face through the milling device, and perform slope distortion correction on the structured light pattern according to the first distortion correction parameter, so that the construction personnel and / or the milling device perform milling operations on the tunnel face according to the instruction of the structured light pattern; The data processing device is used to input the first image data into a pre-trained correction prediction model for processing to obtain the first distortion correction parameter, including: Inputting the first image data into a feature extraction module of the correction prediction model, identifying the positions of at least three target positioning markers in the first image data through the feature extraction module, and obtaining a position feature vector; The position feature vector is input into a regression prediction module of the correction prediction model to obtain the first distortion correction parameter determined by the regression prediction module according to the position feature vector.

Images