Milling and digging boundary control method and system based on AI

Abstract

The invention provides an AI-based milling and digging boundary control method and system, and the method comprises the steps: carrying out the image collection of a target tunnel in a direction away from a tunnel face through a first image collection device disposed on a milling and digging device, and obtaining first image data; and determining a first distortion correction parameter according to at least three target positioning markers in the first image data through a pre-trained correction prediction model, and then projecting a structured light pattern corrected by the first distortion correction parameter towards the tunnel face through projection equipment arranged on the milling and excavating equipment. Therefore, under the condition that the milling and excavating equipment moves to work in the target tunnel, the proper structured light pattern for indicating the milling and excavating boundary can be automatically projected on the tunnel face, and a constructor can more clearly determine the milling and excavating boundary according to the structured light pattern, so that the risk of under-excavation or over-excavation in the milling and excavating process is reduced.

Description

Technical Field

[0001] The present application relates to the field of AI image processing, and more specifically, to an AI-based milling and excavation boundary control method and system. Background Art

[0002] In some tunnel construction operations, milling equipment is used for tunnel excavation to reduce the impact of excavation on surrounding rock formations or ground facilities. During the milling operation, the milling equipment performs milling operations on the face through the milling head. After milling to a certain depth, support is performed, and then further milling is performed. Support and milling operations are performed alternately. In the existing milling scheme, construction personnel operate the rocker arm with a milling head on the milling equipment to perform milling operations on the face. Usually, the extension direction of the rocker arm is consistent with the extension direction of the construction personnel's line of sight, which makes it difficult for construction personnel to judge the extension distance of the rocker arm. As a result, when the rocker arm is extended from the middle of the tunnel to mill the edge of the face, it is easy to cause under-excavation or over-excavation due to inaccurate judgment of the milling depth, and subsequent construction is required for remedial work, affecting construction efficiency. Summary of the invention

[0003] In order to overcome the above-mentioned deficiencies in the prior art, the purpose of this application is to provide an AI-based milling and excavation boundary control method, the method comprising: By using a first image acquisition device disposed on the milling and excavation device, image acquisition is performed in a direction of the target tunnel away from the tunnel face to obtain first image data of at least three target positioning markers including an excavated area of ​​the target tunnel; Inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter; Projecting a structured light pattern indicating a milling boundary toward the tunnel face by means of a projection device disposed on the milling device, and performing slope distortion correction on the structured light pattern according to the first distortion correction parameter; Milling and excavation operations are performed on the tunnel face according to the structured light pattern instructions.

[0004] In some possible implementations, the step of inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter includes: Inputting the first image data into a feature extraction module of the correction prediction model, identifying positions of at least three target positioning markers in the first image data through the feature extraction module, and obtaining a position feature vector; The position feature vector is input into a regression prediction module of the correction prediction model to obtain the first distortion correction parameter determined by the regression prediction module according to the position feature vector.

[0005] In some possible implementations, the method further includes: Acquire equipment parameters of the milling and digging equipment, wherein the milling and digging equipment includes a main body and a cantilever provided with a milling and digging head, and the equipment parameters include an operation distance parameter characterizing a distance range between the main body of the milling and digging equipment and a tunnel face when performing a milling and digging operation; The step of inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter includes: The first image data and the device parameters are input into a pre-trained correction prediction model for processing to obtain first distortion correction parameters.

[0006] In some possible implementations, the step of projecting a structured light pattern for indicating a milling boundary toward the tunnel face by a projection device disposed on the milling device includes: Acquire tunnel excavation parameters of the target tunnel, where the tunnel excavation parameters are used to indicate the size and / or shape of the inner wall of the tunnel that needs to be excavated; A corresponding structured light pattern is generated according to the tunnel excavation parameters, and the structured light pattern is projected toward the tunnel face by a projection device arranged on the milling device.

[0007] In some possible implementations, the method further includes: Acquiring working status data of the milling and digging equipment; According to the working status data, when it is detected that the milling device drives the first image acquisition device and / or the projection device to move or rotate, new first image data is re-acquired and a new structured light pattern is projected.

[0008] In some possible implementations, the structured light pattern includes a grid pattern; and the method further includes: By using a second image acquisition device disposed on the milling device to perform image acquisition toward the tunnel face, second image data including the structured light pattern is obtained; determining a position correction parameter according to the distortion of the structured light pattern at the junction of the tunnel face and the tunnel side wall in the second image data; The projection position of the structured light pattern is adjusted according to the position correction parameter.

[0009] In some possible implementations, the method further includes: The projection colors of different projection positions on the structured light pattern are adjusted according to the positions of the concave-convex distortion of the structured light pattern in the second image data.

[0010] In some possible implementations, the method further includes: Inputting the second image data into a pre-trained content recognition model, and determining, by means of the content recognition model, a first area in the second image data where a milling slag deposit is formed and a second area on the tunnel face where the milling slag deposit is not covered; The step of adjusting the projection colors of different projection positions on the structured light pattern according to the position of the concave-convex distortion of the structured light pattern in the second image data comprises: For the second area, the projection colors of different projection positions on the structured light pattern are adjusted according to the positions of the concave-convex distortion of the structured light pattern in the second image data.

[0011] In some possible implementations, the method further includes: The first image acquisition device disposed on the milling and excavation device performs image acquisition in a direction away from the tunnel face of the sample tunnel to obtain first sample image data of at least three target positioning markers including the excavated area of ​​the target tunnel; Projecting a sample structured light pattern toward the tunnel face by the projection device disposed on the milling device, and adjusting a slope distortion correction parameter of the sample structured light pattern to obtain a first sample correction parameter; The correction prediction model to be trained is trained using a plurality of the first sample images and the corresponding first sample correction parameters, so that the correction prediction model outputs corresponding distortion correction parameters according to the input image data.

[0012] Another object of the present application is to provide an AI-based milling and digging boundary control system, the system comprising a data processing device, a first image acquisition device and a projection device; the first image acquisition device and the projection device are arranged on the milling and digging device; The first image acquisition device is used to perform image acquisition in a direction away from the tunnel face of the target tunnel, and obtain first image data of at least three target positioning markers including the excavated area of ​​the target tunnel; The data processing device is used to input the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter; The data processing device is also used to project a structured light pattern for indicating the milling boundary toward the tunnel face through the milling device, and perform slope distortion correction on the structured light pattern according to the first distortion correction parameter, so that construction personnel and / or the milling device perform milling operations on the tunnel face according to the instructions of the structured light pattern.

[0013] Compared with the prior art, this application has the following beneficial effects: The AI-based milling and excavation boundary control method and system provided in the present application obtains first image data by performing image acquisition in the direction of the target tunnel away from the face through a first image acquisition device arranged on the milling and excavation equipment, and determines the first distortion correction parameter according to at least three target positioning markers in the first image data through a pre-trained correction prediction model, and then projects the structured light pattern corrected by the first distortion correction parameter toward the face through a projection device arranged on the milling and excavation equipment. In this way, when the milling and excavation equipment is moving in the target tunnel, a suitable structured light pattern indicating the milling and excavation boundary can also be automatically projected on the face, and the construction personnel can more clearly determine the milling and excavation boundary according to the structured light pattern, thereby reducing the risk of under-excavation or over-excavation during the milling and excavation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A schematic diagram of an AI-based milling and excavation boundary control system provided in an embodiment of the present application; Figure 2 One of the application scenario schematic diagrams of the AI-based milling and excavation boundary control system provided in the embodiment of the present application; Figure 3 A schematic diagram of the steps of the AI-based milling and excavation boundary control method provided in an embodiment of the present application; Figure 4 The second schematic diagram of the application scenario of the AI-based milling and excavation boundary control system provided in the embodiment of the present application; Figure 5 The third schematic diagram of the application scenario of the AI-based milling and excavation boundary control system provided in the embodiment of the present application; Figure 6 A schematic diagram of a data device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0017] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0018] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0019] In the description of the present application, it should be noted that the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0020] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0021] See also Figure 1 , Figure 1 A schematic diagram of an AI-based milling and excavation boundary control system provided in this embodiment, the system may include a data processing device 100, a first image acquisition device 200 and a projection device 300.

[0022] See also Figure 2 , Figure 2 The schematic diagram of the application scenario of the solution provided in this embodiment is that the milling and digging equipment 600 may be a cantilever milling and digging equipment 600 with a milling and digging head, and the milling and digging equipment 600 may include a body and a cantilever. The first image acquisition device 200 and the projection device 300 may be arranged on the milling and digging equipment 600. The first image acquisition device 200 may be arranged on the body, and the acquisition direction of the first image acquisition device 200 may deviate from the extension direction of the cantilever, and the projection device 300 may be arranged on the body, and the projection direction of the projection device 300 may be along the extension direction of the cantilever.

[0023] For example, the body of the milling and digging equipment 600 is provided with a cab, and during the operation of the milling and digging equipment 600, the posture seat in the cab is usually facing the tunnel face 700. The projection direction of the projection device 300 can be consistent with the direction of the driver's seat, and the image acquisition direction of the first image acquisition device 200 can be opposite to the direction of the driver's seat.

[0024] The data processing device 100, the first image acquisition device 200 and the projection device 300 cooperate with each other to implement the AI-based milling and excavation boundary control method provided in this embodiment.

[0025] Specifically, see Figure 3 The AI-based milling and excavation boundary control method may include the following steps.

[0026] Step S110, performing image acquisition in a direction away from the tunnel face 700 of the target tunnel by a first image acquisition device arranged on the milling and excavation device 600, and obtaining first image data of at least three target positioning markers 900 including the excavated area of ​​the target tunnel.

[0027] See also Figure 4 In this embodiment, the target tunnel may include an excavated area and an area to be excavated, wherein the excavated area may be an area where excavation or support has been completed, and the inner wall of the tunnel in this area will not be subjected to a large number of construction operations in a short period of time. The target positioning marker 900 may be respectively set at at least three different specific locations in the area. The target positioning marker 900 may be an indicator light of a special color. For example, the target positioning marker 900 may be respectively set at the arch, the middle of the tunnel wall on both sides, and the bottom of the tunnel side walls on both sides.

[0028] In this embodiment, when the milling and excavation equipment 600 performs milling and excavation operations toward the tunnel face 700, the first image acquisition device 200 arranged on the milling and excavation equipment 600 can perform image acquisition toward the excavated area, and the acquired image can include at least three of the above-mentioned target positioning markers 900.

[0029] It should be noted that, as the excavation of the tunnel face 700 progresses, the position of the target positioning marker 900 can also be adjusted accordingly, but the advancement speed of the target positioning marker 900 can be much lower than the advancement speed of the tunnel face 700. For example, the milling and digging equipment 600 alternately performs milling and digging and support at a speed of 1 meter per excavation, and after 50 milling and digging advances (i.e., 50 meters), it advances 50 meters along the extension direction of the tunnel to reinstall or set the target positioning marker 900. In this way, it can be ensured that the image captured by the first image acquisition device 200 can clearly capture the target positioning marker 900, and it can also avoid the frequent movement of the target positioning marker 900 affecting the construction progress.

[0030] It should be noted that, in the present embodiment, the setting position of each target positioning marker 900 should be relatively fixed, so that the relative position relationship between the milling and excavation equipment 600 and each target positioning marker 900 can be determined in subsequent steps.

[0031] Step S120: input the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter.

[0032] Step S130 , projecting a structured light pattern for indicating a milling boundary toward the tunnel face 700 by means of the projection device 300 disposed on the milling device 600 , and performing slope distortion correction on the structured light pattern according to the first distortion correction parameter.

[0033] Step S140 , performing milling operations on the tunnel face 700 according to the structured light pattern instruction.

[0034] In this embodiment, in step S130, when the milling device 600 performs milling operation toward the tunnel face 700, the projection device 300 disposed on the milling device 600 may project a structured light pattern on the tunnel face 700. The structured light pattern may include a stripe pattern or a grid pattern.

[0035] When the structured light pattern is projected on a relatively flat plane, the stripes or grids therein will not be distorted. However, when it is projected on an uneven plane or at the junction of surfaces, the stripes and grids therein will be distorted.

[0036] In this embodiment, when the structured light pattern is projected at the junction of the tunnel face 700 and the tunnel side wall, the distortion of the structured light pattern can clearly show the junction position of the current tunnel face 700 and the side wall. In addition, different morphological lines or lines of different colors can be projected in the structured light pattern to indicate the desired milling and excavation boundary indication line. In this way, in step S140, the construction personnel operating the milling and excavation equipment 600 can see the desired milling and excavation boundary indication line from the structured light pattern, and can clearly see the current milling and excavation boundary according to the distortion of the structured light pattern, so that the milling and excavation operation can be better performed accordingly.

[0037] In this example, see Figure 5 , during the milling operation of the milling device 600, the position of the milling device 600 relative to the tunnel face 700 may move, and the milling device 600 does not always perform the milling operation perpendicularly toward the tunnel face 700. When the projection device 300 projects the structured light pattern toward the tunnel face 700, the projection angle and projection direction will also change accordingly. Different projection angles and projection directions will cause the projected structured light pattern to produce slope distortion.

[0038] Therefore, before performing the projection, step S120 may be performed first. In step S120, the collected first image data may be input into a pre-trained correction prediction model for processing. The correction prediction model may determine the corresponding first distortion correction parameter according to the first image data.

[0039] Specifically, step S120 may include the following sub-steps.

[0040] Step S121, inputting the first image data into the feature extraction module of the correction prediction model, identifying the positions of at least three target positioning markers 900 in the first image data through the feature extraction module, and obtaining a position feature vector.

[0041] That is, the feature extraction module can perform image content recognition on the first image data, and determine the positions of at least three of the target positioning markers 900 in the first image data as the position feature vector.

[0042] Specifically, when the milling and digging device 600 is in different positions, the shooting angle or shooting direction of the first image acquisition device 200 is different. In this case, the relative position of the captured target positioning marker 900 in the first image data is also different. Therefore, the position of each target positioning marker 900 in the first image data can also represent the relative position relationship between the milling and digging device 600 and each target positioning marker 900, and the relative position of each target positioning marker 900 and the face 700 is fixed, so the position feature vector obtained in step S121 can also represent the relative position relationship between the milling and digging device 600 and the face 700.

[0043] Step S122: input the position feature vector into the regression prediction module of the correction prediction model to obtain the first distortion correction parameter determined by the regression prediction module according to the position feature vector.

[0044] Specifically, after the position feature vector is determined in step S121, a regression prediction can be performed according to the position feature vector in step S122 to determine the corresponding first distortion correction parameter. The first distortion correction parameter can be used to perform slope distortion correction on the structured light image in step S130, thereby offsetting the distortion caused by the projection device 300 tilting toward the palm face 700 during the projection process.

[0045] Based on the above design, in the solution provided in this embodiment, the first image acquisition device 200 arranged on the milling and excavation equipment 600 performs image acquisition in the direction away from the tunnel face 700 of the target tunnel to obtain the first image data, and the first distortion correction parameter is determined according to at least three target positioning markers 900 in the first image data through the pre-trained correction prediction model, and then the structured light pattern corrected by the first distortion correction parameter is projected toward the tunnel face 700 by the projection device 300 arranged on the milling and excavation equipment 600. In this way, when the milling and excavation equipment 600 is moving in the target tunnel, a suitable structured light pattern indicating the milling and excavation boundary can also be automatically projected on the tunnel face 700, and the construction personnel can more clearly determine the milling and excavation boundary according to the structured light pattern, thereby reducing the risk of under-excavation or over-excavation in the milling and excavation process.

[0046] In some possible implementations, the method provided in this embodiment may further include step S210.

[0047] Step S210, obtaining equipment parameters of the milling and digging equipment 600, wherein the milling and digging equipment 600 includes a main body and a cantilever provided with a milling and digging head, and the equipment parameters include an operating distance parameter characterizing the distance range between the main body of the milling and digging equipment 600 and the tunnel face 700 when performing milling and digging operations.

[0048] Specifically, when the milling head of the same milling equipment 600 rotates to perform milling operations, the body of the milling equipment 600 usually does not move. The extension and movement of the cantilever drive the milling head to perform milling operations on different positions. Therefore, the projected structured light pattern does not need to be adjusted during this process.

[0049] However, different milling and digging equipment 600 have different cantilever lengths, so the distance between the milling and digging equipment 600 and the tunnel face 700 will also be different, which will lead to different projection distances of the projection device 300. In order to avoid projection image distortion or change in projection image range caused by different projection distances, in step S130, the first image data and the device parameters can be input into a pre-trained correction prediction model for processing to obtain the first distortion correction parameters. That is, the first distortion correction parameters are determined based on the first image data and the device parameters.

[0050] In some possible implementations, step S130 may further include the following sub-steps.

[0051] Step S131, obtaining tunnel excavation parameters of the target tunnel, wherein the tunnel excavation parameters are used to indicate the size and / or shape of the inner wall of the tunnel that needs to be excavated.

[0052] In this embodiment, the design requirements of the target tunnel may be acquired in advance, and tunnel excavation parameters may be entered according to the design requirements.

[0053] Step S132 , generating a corresponding structured light pattern according to the tunnel excavation parameters, and projecting the structured light pattern toward the tunnel face 700 by means of the projection device 300 disposed on the milling device 600 .

[0054] In this embodiment, a corresponding structured light pattern can be generated according to the tunnel excavation parameters. For example, in a striped or grid-shaped structured light pattern, a milling boundary indicator line that is consistent with the tunnel excavation parameters is formed by projecting different lines or different colors.

[0055] It should be noted that in this embodiment, in order to better display the current milling and excavation boundary through the distortion of the structured light pattern, the coverage area of ​​the structured light pattern needs to be larger than the area of ​​the tunnel face 700, and at least part of the structured light pattern needs to cover the junction of the tunnel face 700 and the tunnel side wall, or cover the junction of the tunnel face 700 and the ground.

[0056] In some possible implementations, the working status data of the milling and digging equipment 600 can be obtained first, and then, based on the working status data, when it is detected that the milling and digging equipment 600 drives the first image acquisition device 200 and / or the projection device 300 to move or rotate, new first image data can be recaptured and a new structured light pattern can be projected.

[0057] Specifically, during the process of the milling head of the milling equipment 600 rotating to perform milling, the body of the milling equipment 600 usually does not move or rotate, and only the cantilever moves to drive the milling head to perform milling. Therefore, the projected structured light pattern at this stage does not need to perform new slope distortion correction.

[0058] After completing the milling of a small area, the milling device 600 moves or rotates at a certain angle to mill another small area. In this case, the projection device 300 moves or rotates with the milling device 600. Therefore, whenever it is detected that the milling device 600 drives the first image acquisition device 200 and / or the projection device 300 to move or rotate, the new first image data is recaptured and the new structured light pattern is projected, that is, the projection of the new structured light image and the slope distortion correction action are re-executed to ensure the accuracy of the projected structured light image.

[0059] In some possible implementations, the structured light pattern includes a grid pattern. The solution provided in this embodiment may also include the following steps.

[0060] Step S310: A second image acquisition device disposed on the milling device 600 performs image acquisition toward the tunnel face 700 to obtain second image data including the structured light pattern.

[0061] Step S320: determining a position correction parameter according to the distortion of the structured light pattern in the second image data at the junction of the tunnel face 700 and the tunnel side wall.

[0062] Step S330: adjusting the projection position of the structured light pattern according to the position correction parameter.

[0063] Specifically, in this embodiment, in step S120, the correction prediction model can determine the position feature vector that can roughly characterize the relative position relationship between the milling and digging equipment 600 and the tunnel face 700 according to the position of each target positioning marker 900 in the first image data. However, for different milling and digging equipment 600, the setting position of the projection device 300 or the overall height of the milling and digging equipment 600 may be different.

[0064] Furthermore, since the structured light pattern may be distorted at the boundary of the current tunnel face 700 , in step S310 , second image data of the tunnel face 700 after the structured light pattern is projected onto the tunnel face 700 may be acquired by a second image acquisition device.

[0065] In step S320 and step S330, the position of the projected structured light pattern can be adjusted according to the distorted position of the structured light pattern in the second image data at the boundary of the current tunnel face 700. For example, the projected structured light pattern can be moved or scaled so that the milling boundary indicator line in the structured light pattern is aligned with the position where the distortion is currently generated at the boundary of the tunnel face 700. In this way, it is ensured that the pattern projected by the projection device 300 can be accurately aligned with the tunnel face 700.

[0066] In some possible implementations, the method provided in this embodiment may further include step S340.

[0067] Step S340 , adjusting the projection colors of different projection positions on the structured light pattern according to the positions of the concave-convex distortion of the structured light pattern in the second image data.

[0068] Specifically, during the milling process, there is an unevenness between the area that has been milled and the area that has not yet been milled. In order to better display the area that has not yet been milled, when projecting the grid of the structured light pattern, the color of the line of the grid projected to the position where the concave-convex distortion occurs in the second image can be adjusted to indicate the situation of the corresponding area.

[0069] For example, the grid lines at the positions that are prominent relative to other areas are projected in red, and the grid lines at other flat areas (areas where the grid is not distorted) are projected in green.

[0070] For some possible implementations, see again Figure 2 During the milling process, the rock layer broken by the milling head will fall to the bottom of the tunnel face 700 to form a milling debris deposit 800. In order to avoid the influence of the milling debris deposit 800 on the identification of the concave-convex distortion position in the second image, in this embodiment, the second image data can be first input into a pre-trained content recognition model, and the first area formed by the milling debris deposit 800 in the second image data and the second area of ​​the tunnel face 700 not covered by the milling debris deposit 800 are determined by the content recognition model.

[0071] Then, for the second area, the projection colors of different projection positions on the structured light pattern are adjusted according to the positions of the concave-convex distortion of the structured light pattern in the second image data.

[0072] In some possible implementations, this embodiment also provides a training step for correcting the prediction model, which may specifically include the following steps.

[0073] Step S510: The first image acquisition device 200 disposed on the milling and excavation device 600 performs image acquisition in a direction of the sample tunnel away from the tunnel face 700 to obtain first sample image data of at least three target positioning markers 900 including the excavated area of ​​the target tunnel.

[0074] Step S520 , projecting a sample structured light pattern toward the tunnel face 700 by the projection device 300 disposed on the milling device 600 , and adjusting the slope distortion correction parameters of the sample structured light pattern to obtain first sample correction parameters.

[0075] Step S530: Use the plurality of first sample images and the corresponding first sample correction parameters to train the correction prediction model to be trained, so that the correction prediction model outputs corresponding distortion correction parameters according to the input image data.

[0076] Specifically, in step S510 and step S520, when the milling device 600 is in the working position, the first sample image data can be collected by the first image acquisition device 200. At the same time, the sample structured light pattern is projected by the projection device 300, and then the distortion correction parameters of the projection device 300 are manually adjusted so that there is no correction distortion in the projection of the sample structured light pattern on the tunnel face 700, and the currently used distortion correction parameters are used as the first sample correction parameters.

[0077] Next, in step S530, the correction prediction model to be trained is trained using a plurality of the first sample images and the corresponding first sample correction parameters. The correction prediction model can output a second sample correction parameter according to the input first sample image data, and then adjust the network parameters of the correction prediction model according to the first sample correction parameter and the second sample correction parameter to reduce the gap between the first sample correction parameter and the second sample correction parameter.

[0078] After the number of times of training using different samples reaches a preset threshold, or the difference between the first sample correction parameter and the second sample correction parameter is less than a set threshold, the trained correction prediction model is obtained.

[0079] See also Figure 6 , Figure 6 yes Figure 1 The data processing device 100 is a block diagram showing a data processing device 100. The data processing device 100 comprises an AI-based milling and excavation boundary control device 110, a machine-readable storage medium 120, and a processor 130.

[0080] The components of the machine-readable storage medium 120 and the processor 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The AI-based milling and digging boundary control device 110 includes at least one software function module that can be stored in the form of software or firmware in the machine-readable storage medium 120 or solidified in the operating system (OS) of the data processing device 100. The processor 130 is used to execute the executable modules stored in the machine-readable storage medium 120, such as the software function modules and computer programs included in the AI-based milling and digging boundary control device 110.

[0081] The machine-readable storage medium 120 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The machine-readable storage medium 120 is used to store a program, and the processor 130 executes the program / executable AI-based milling and excavation boundary control method provided in this embodiment after receiving an execution instruction.

[0082] The processor 130 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0083] In summary, the AI-based milling and excavation boundary control method and system provided in the present application obtains first image data by performing image acquisition in the direction of the target tunnel away from the face through a first image acquisition device arranged on the milling and excavation equipment, and determines the first distortion correction parameter according to at least three target positioning markers in the first image data through a pre-trained correction prediction model, and then projects the structured light pattern corrected by the first distortion correction parameter toward the face through a projection device 300 arranged on the milling and excavation equipment. In this way, when the milling and excavation equipment is moving in the target tunnel, a suitable structured light pattern indicating the milling and excavation boundary can also be automatically projected on the face, and the construction personnel can more clearly determine the milling and excavation boundary according to the structured light pattern, thereby reducing the risk of under-excavation or over-excavation during the milling and excavation process.

[0084] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0085] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0086] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0087] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0088] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A milling and excavation boundary control method based on AI, characterized in that: The method comprises: By using a first image acquisition device disposed on the milling and excavation device, image acquisition is performed in a direction of the target tunnel away from the tunnel face to obtain first image data of at least three target positioning markers including an excavated area of ​​the target tunnel; Inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter; Projecting a structured light pattern indicating a milling boundary toward the tunnel face by means of a projection device disposed on the milling device, and performing slope distortion correction on the structured light pattern according to the first distortion correction parameter; Milling and excavation operations are performed on the tunnel face according to the structured light pattern instructions.

2. The method according to claim 1, characterized in that The step of inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter includes: Inputting the first image data into a feature extraction module of the correction prediction model, identifying positions of at least three target positioning markers in the first image data through the feature extraction module, and obtaining a position feature vector; The position feature vector is input into a regression prediction module of the correction prediction model to obtain the first distortion correction parameter determined by the regression prediction module according to the position feature vector.

3. The method according to claim 1, characterized in that The method further comprises: Acquire equipment parameters of the milling and digging equipment, wherein the milling and digging equipment includes a main body and a cantilever provided with a milling and digging head, and the equipment parameters include an operation distance parameter characterizing a distance range between the main body of the milling and digging equipment and a tunnel face when performing a milling and digging operation; The step of inputting the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter includes: The first image data and the device parameters are input into a pre-trained correction prediction model for processing to obtain first distortion correction parameters.

4. The method according to claim 1, characterized in that: The step of projecting a structured light pattern for indicating a milling boundary toward the tunnel face by means of a projection device disposed on the milling device comprises: Acquire tunnel excavation parameters of the target tunnel, where the tunnel excavation parameters are used to indicate the size and / or shape of the inner wall of the tunnel that needs to be excavated; A corresponding structured light pattern is generated according to the tunnel excavation parameters, and the structured light pattern is projected toward the tunnel face by a projection device arranged on the milling device.

5. The method according to claim 1, characterized in that The method further comprises: Acquiring working status data of the milling and digging equipment; According to the working status data, when it is detected that the milling device drives the first image acquisition device and / or the projection device to move or rotate, new first image data is re-acquired and a new structured light pattern is projected.

6. The method according to claim 1, characterized in that The structured light pattern comprises a grid pattern; and the method further comprises: By using a second image acquisition device disposed on the milling device to perform image acquisition toward the tunnel face, second image data including the structured light pattern is obtained; determining a position correction parameter according to the distortion of the structured light pattern at the junction of the tunnel face and the tunnel side wall in the second image data; The projection position of the structured light pattern is adjusted according to the position correction parameter.

7. The method according to claim 6, characterized in that The method further comprises: The projection colors of different projection positions on the structured light pattern are adjusted according to the positions of the concave-convex distortion of the structured light pattern in the second image data.

8. The method according to claim 7, characterized in that The method further comprises: Inputting the second image data into a pre-trained content recognition model, and determining, by means of the content recognition model, a first area in the second image data where a milling slag deposit is formed and a second area on the tunnel face where the milling slag deposit is not covered; The step of adjusting the projection colors of different projection positions on the structured light pattern according to the position of the concave-convex distortion of the structured light pattern in the second image data comprises: For the second area, the projection colors of different projection positions on the structured light pattern are adjusted according to the positions of the concave-convex distortion of the structured light pattern in the second image data.

9. The method according to claim 1, characterized in that: The method further comprises: The first image acquisition device disposed on the milling and excavation device performs image acquisition in a direction away from the tunnel face of the sample tunnel to obtain first sample image data of at least three target positioning markers including the excavated area of ​​the target tunnel; Projecting a sample structured light pattern toward the tunnel face by the projection device disposed on the milling device, and adjusting a slope distortion correction parameter of the sample structured light pattern to obtain a first sample correction parameter; The correction prediction model to be trained is trained using a plurality of the first sample images and the corresponding first sample correction parameters, so that the correction prediction model outputs corresponding distortion correction parameters according to the input image data.

10. An AI-based milling and excavation boundary control system, characterized in that: The system comprises a data processing device, a first image acquisition device and a projection device; the first image acquisition device and the projection device are arranged on the milling and digging device; The first image acquisition device is used to perform image acquisition in a direction away from the tunnel face of the target tunnel, and obtain first image data of at least three target positioning markers including the excavated area of ​​the target tunnel; The data processing device is used to input the first image data into a pre-trained correction prediction model for processing to obtain a first distortion correction parameter; The data processing device is also used to project a structured light pattern for indicating the milling boundary toward the tunnel face through the milling device, and perform slope distortion correction on the structured light pattern according to the first distortion correction parameter, so that construction personnel and / or the milling device perform milling operations on the tunnel face according to the instructions of the structured light pattern.

Images