Mechanical Excavation Method and Device for Image Recognition

Abstract

The present application provides a mechanical excavation method and device for image recognition. The first video image of the heading face during the operation of the milling excavation equipment is collected by an image acquisition device, and the milling excavation slag falling speed characteristics during the milling excavation process are determined through image recognition by a pre-established milling excavation strategy prediction model. Together with the pre-recorded rock stratum exploration data, a suitable predicted value of the milling excavation step depth is determined. In this way, the milling excavation step depth can be adjusted in a timely manner when facing different rock stratum states during the milling excavation process, and the construction efficiency can be improved while ensuring construction safety.

Description

Technical Field

[0001] This application relates to the field of AI-based image recognition processing, and more specifically, to a mechanical excavation method and device for image recognition. Background Art

[0002] In some tunnel construction operations, milling equipment is used for tunnel excavation to reduce the impact of excavation operations on surrounding rock formations or surface facilities. During the milling operation, the milling equipment performs milling operations on the heading face through a milling head. After milling to a certain depth, support is carried out, and then further milling is performed, with the support and milling operations alternating. However, the rock formation states at different positions may be different, and the rock formation state directly affects the milling step depth. If the milling step depth is too small, the alternation of support and milling operations is too frequent, affecting the construction speed; if the milling depth is too large, there may be construction safety hazards due to untimely support. How to timely and accurately adaptively adjust the milling step depth has become an urgent problem to be solved. Summary of the Invention

[0003] In order to overcome the above deficiencies in the prior art, the purpose of this application is to provide a mechanical excavation method for image recognition, and the method includes:

[0004] Obtain a first video image of the heading face during the operation of the milling equipment through an image acquisition device;

[0005] Process the first video image through the milling slag falling speed feature extraction module of a pre-trained milling strategy prediction model to obtain a milling slag falling speed feature;

[0006] Fuse the pre-recorded rock formation exploration data with the milling slag falling speed feature to obtain a fused feature;

[0007] Input the fused feature into the step depth prediction module of the milling strategy prediction model to obtain a milling step depth prediction value input to the step depth prediction module;

[0008] Send the milling step depth prediction value to the milling operator and / or the milling equipment to instruct the milling operator and / or the milling equipment to perform milling operations according to the milling step depth prediction value.

[0009] In some possible implementation manners, the step of processing the first video image through the milling slag falling speed feature extraction module of a pre-trained milling strategy prediction model to obtain a milling slag falling speed feature includes:

[0010] Obtain multiple video image frames of the first video image;

[0011] The milling and excavation slag dropping speed feature extraction module respectively performs content recognition on a plurality of the video images to determine the slag deposits in each frame of the video images;

[0012] Determine the milling and excavation slag dropping speed feature according to the difference of the slag deposits between a plurality of adjacent frames of the video images within a preset time period.

[0013] In some possible implementation manners, the step of the milling and excavation slag dropping speed feature extraction module respectively performing content recognition on a plurality of the video images to determine the slag deposits in each frame of the video images includes:

[0014] The milling and excavation slag dropping speed feature extraction module respectively performs content recognition on a plurality of the video images to determine the slag deposits in each frame of the video images, and determines the working state of the milling head of the milling and excavation equipment; the working state includes an operating state and a non-operating state;

[0015] The step of determining the milling and excavation slag dropping speed feature according to the difference of the slag deposits between a plurality of adjacent frames of the video images within a preset time period includes:

[0016] When it is detected that the milling head is in the working state for a preset time period, determine the milling and excavation slag dropping speed feature according to the difference of the slag deposits between a plurality of adjacent frames of the video images within the preset time period.

[0017] In some possible implementation manners, the method further includes:

[0018] Process the first video image through the milling position feature extraction module of a pre-trained milling strategy prediction model to obtain the operation position feature of the milling head of the milling and excavation equipment relative to the face;

[0019] The step of determining the milling and excavation slag dropping speed feature according to the difference of the slag deposits between a plurality of adjacent frames of the video images within a preset time period includes:

[0020] Determine the milling and excavation slag dropping speed feature according to the difference of the slag deposits between a plurality of adjacent frames of the video images within a preset time period and the operation position feature.

[0021] In some possible implementation manners, the method further includes:

[0022] Obtain the working parameters of the milling head of the milling and excavation equipment, where the working parameters of the milling head include at least one of milling width, milling head diameter, number of cutter heads, current rotation speed, and torque;

[0023] The step of fusing the pre-recorded rock stratum exploration data with the milling and excavation slag dropping speed feature to obtain a fusion feature includes:

[0024] Fuse the pre - entered rock stratum exploration data, the working parameters of the milling head, and the milling slag falling speed characteristics to obtain fusion characteristics.

[0025] In some possible implementation manners, the method further includes:

[0026] Input the fusion characteristics into the milling parameter adjustment module of the milling strategy prediction model for processing, and obtain the milling parameter adjustment result output by the milling parameter adjustment module;

[0027] Apply the milling parameter adjustment result to the milling operation of the milling head.

[0028] In some possible implementation manners, the method further includes:

[0029] Determine the corresponding dust - suppression operation control parameters according to the milling slag falling speed characteristics;

[0030] Control the dust - suppression equipment to perform the dust - suppression operation according to the dust - suppression operation control parameters.

[0031] In some possible implementation manners, the method further includes:

[0032] Input the fusion characteristics into the slag falling speed warning module of the milling strategy prediction model for processing, and obtain the slag falling speed prediction threshold;

[0033] Detect whether the slag falling speed corresponding to the milling slag falling speed characteristics reaches the slag falling speed prediction threshold;

[0034] If so, output a risk warning prompt.

[0035] Another object of the present application is to provide a mechanical excavation device for image recognition, and the mechanical excavation device for image recognition includes:

[0036] A video acquisition module, configured to acquire the first video image of the heading face during the operation of the milling equipment collected by the image acquisition device;

[0037] A feature extraction module, configured to process the first video image through the milling slag falling speed feature extraction module of the pre - trained milling strategy prediction model to obtain the milling slag falling speed characteristics;

[0038] A feature fusion module, configured to fuse the pre - entered rock stratum exploration data with the milling slag falling speed characteristics to obtain fusion characteristics;

[0039] A step - by - step prediction module, configured to input the fusion characteristics into the step - by - step depth prediction module of the milling strategy prediction model to obtain the milling step - by - step depth prediction value input by the step - by - step depth prediction module;

[0040] The operation instruction module is used to send the predicted value of the milling step depth to the milling operator and / or the milling equipment, so as to instruct the milling operator and / or the milling equipment to perform the milling operation according to the predicted value of the milling step depth.

[0041] Compared with the prior art, the present application has the following beneficial effects:

[0042] The present application provides a mechanical excavation method and device based on image recognition. The first video image of the face during the operation of the milling equipment is collected by an image acquisition device, and the image recognition is carried out through a pre-set milling strategy prediction model to determine the characteristics of the milling slag falling speed during the milling process. Together with the pre-entered rock stratum exploration data, the appropriate predicted value of the milling step depth is determined. In this way, the milling step depth can be adjusted in a timely manner when facing different rock stratum states during the milling process, and the construction efficiency can be improved while ensuring construction safety. Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 It is a schematic diagram of the mechanical excavation system based on image recognition provided by the embodiment of the present application;

[0045] Figure 2 It is a schematic diagram of the step flow of the mechanical excavation system method based on image recognition provided by the embodiment of the present application;

[0046] Figure 3 It is a schematic diagram of the milling scene provided by the embodiment of the present application;

[0047] Figure 4 It is a schematic diagram of the data processing device provided by the embodiment of the present application;

[0048] Figure 5 It is a schematic diagram of the functional modules of the mechanical excavation device based on image recognition provided by the embodiment of the present application. Detailed Embodiments

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Apparently, the described embodiments are some, but not all, of the embodiments of this application. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application that is claimed, but merely represents selected embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts fall within the scope of protection of this application.

[0051] It should be noted that: Similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0052] In the description of this application, it should be noted that the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0053] In the description of this application, it should also be noted that unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "couple" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific situations.

[0054] See Figure 1 , Figure 1 which is a schematic diagram of a mechanical excavation system for image recognition provided for this embodiment. The system can include a data processing device 100 and at least one image acquisition device 200. The first video image of the face during the operation of the milling excavation device is acquired by the image acquisition device, and the milling excavation slag falling speed feature during the milling excavation process is determined through image recognition by a pre-established milling excavation strategy prediction model, and the appropriate milling excavation step depth prediction value is determined in combination with the pre-entered rock stratum exploration data.

[0055] Specifically, the data processing device 100 can be used to execute a mechanical excavation method for image recognition provided for this embodiment. Please see Figure 2 , and this method can include the following steps.

[0056] Step S110, obtain a first video image of the heading face during the operation of the milling excavation equipment through an image acquisition device.

[0057] In this embodiment, the image acquisition device can be arranged on the milling excavation equipment or on the top of the tunnel. The image acquisition device can acquire the first video image of the heading face during the operation of the milling excavation equipment, that is, acquire the image of the rock layer being milled.

[0058] Step S120, process the first video image through the milling excavation slag falling speed feature extraction module of the pre-trained milling excavation strategy prediction model to obtain the milling excavation slag falling speed feature.

[0059] Please refer to Figure 3 , Figure 3 FIG. is a schematic diagram of the milling excavation process of the milling excavation equipment 600. Among them, the milling excavation equipment 600 can be a cantilever type excavation equipment provided with a milling head. The milling excavation equipment can mill the rock layer of the heading face 700 through the milling head. During the milling excavation process, the milling head rotates to break the rock layer, and the broken rock layer slag will fall downward and accumulate at the bottom of the heading face to form a slag accumulation 800.

[0060] In this embodiment, the data processing device 100 can receive the first video image collected by the image acquisition device 200, input the first video image into the pre-trained milling excavation strategy prediction model, and process the first video image through the milling excavation slag falling speed feature extraction module of the milling excavation strategy prediction model to obtain the milling excavation slag falling speed feature representing the slag falling speed during the milling excavation process.

[0061] Among them, the milling excavation slag falling speed feature can, to a certain extent, represent the rock layer state at the current excavation location. For example, when the rotation speed of the milling head is the same, the speed of the slag falling at the position where the rock strength is relatively large during milling excavation is relatively small, and the speed of the slag falling at the position where the rock strength is relatively small during milling excavation is relatively large.

[0062] Step S130, fuse the pre-recorded rock layer exploration data with the milling excavation slag falling speed feature to obtain a fused feature.

[0063] In this embodiment, during the preliminary exploration or construction of the tunnel, holes are drilled in the rock layer for sampling to obtain the rock layer exploration data. The rock layer exploration data can include data such as the type, elastic modulus, Poisson's ratio, and rock layer strength of the rock layer at different positions.

[0064] In this embodiment, the rock layer exploration data can be fused with the milling excavation slag falling speed feature. For example, the milling excavation slag falling speed feature and the rock layer exploration data at the same position are spliced to form the fused feature.

[0065] Step S140: Input the fusion feature into the step depth prediction module of the milling excavation strategy prediction model to obtain the predicted value of the milling excavation step depth input by the step depth prediction module.

[0066] In this embodiment, considering that the rock stratum exploration data can only characterize the general characteristics of the rock stratum, and there may be certain differences in the rock stratum conditions at different positions. Therefore, based on the rock stratum exploration data, combined with the currently extracted milling excavation slag falling speed feature, a suitable predicted value of the milling excavation step depth is obtained through prediction.

[0067] For example, in one example, a basic step depth can be determined first according to the feature representing the rock stratum type in the fusion feature, and then an adjustment step depth can be determined according to the feature representing the milling excavation slag falling speed in the fusion feature. The basic step depth and the adjustment step depth are weighted and summed to obtain the predicted value of the milling excavation step depth.

[0068] For another example, in another example, the fusion feature can be input into the depth prediction module for feature fitting and regression prediction to obtain the output predicted value of the milling excavation step depth.

[0069] Step S150: Send the predicted value of the milling excavation step depth to the milling excavation operator and / or the milling excavation equipment to instruct the milling excavation operator and / or the milling excavation equipment to perform the milling excavation operation according to the predicted value of the milling excavation step depth.

[0070] In this embodiment, after obtaining the predicted value of the milling excavation step depth, it can be sent to the milling excavation operator so that the milling excavation equipment is controlled to perform the milling excavation operation according to the predicted value of the milling excavation step depth, or the predicted value of the milling excavation step depth can be sent to the milling excavation equipment so that it automatically performs the milling excavation operation according to the predicted value of the milling excavation step depth.

[0071] Based on the above design, in the method provided in this application, the first video image of the tunnel face during the operation of the milling excavation equipment is collected by the image acquisition device, and the milling excavation slag falling speed feature during the milling excavation process is determined through image recognition by the pre-established milling excavation strategy prediction model. Combined with the pre-entered rock stratum exploration data, a suitable predicted value of the milling excavation step depth is determined. In this way, the milling excavation step depth can be adjusted in a timely manner in the face of different rock stratum states during the milling excavation process, and the construction efficiency can be improved while ensuring construction safety.

[0072] In some possible implementation manners, step S120 may include the following sub-steps.

[0073] Step S121: Obtain multiple video image frames of the first video image.

[0074] Step S122: The milling and slag - falling speed feature extraction module respectively performs content recognition on multiple video images to determine the slag accumulation in each video image frame.

[0075] Step S123: Determine the milling and slag - falling speed feature according to the difference of the slag accumulation between multiple adjacent video image frames within a preset time period.

[0076] Specifically, in this embodiment, the milling and slag - falling speed feature extraction module can be configured to identify the slag accumulation in the first video image and determine the milling and slag - falling speed feature according to the change of the slag accumulation in multiple video image frames.

[0077] Optionally, in this embodiment, multiple video image frames can be obtained at the same sampling interval, and then the difference value between every two adjacent video image frames of the slag accumulation is obtained. The difference value can be the coverage range of the identified slag accumulation or the slag volume determined according to the coverage range of the slag accumulation. Then, the milling and slag - falling speed feature is determined according to the average value of the difference values of the slag accumulation between multiple video image frames.

[0078] Furthermore, in step S122, the milling and slag - falling speed feature extraction module can respectively perform content recognition on multiple video images to determine the slag accumulation in each video image frame, and at the same time, the working state of the milling head of the milling equipment can also be determined. The working state includes an operating state and a non - operating state.

[0079] For example, when it is recognized that the milling head is in a rotating state and the slag accumulation increases, the working state is determined to be the operating state. When it is recognized that the milling head is not rotating and / or the milling head is rotating but the slag accumulation does not increase, the working state is determined to be the non - working state.

[0080] In step S123, when it is detected that the milling head is in the working state for a preset time period, the milling and slag - falling speed feature is determined according to the difference of the slag accumulation between multiple adjacent video image frames within this preset time period.

[0081] Specifically, within a short period of time when the milling head just comes into contact with the rock formation of the heading face, it is not yet in a stable milling state (i.e., in a starting state). At this time, the milling slag falling speed will be significantly less than the milling slag falling speed during stable milling. In order to reduce the influence of the starting state on the recognition of the milling slag falling speed and avoid the influence of the non-working state on the recognition of the milling slag falling speed, in this embodiment, when it is detected that the milling head has been in the working state for a preset duration, it can be considered that the milling operation is in a stable milling state, and then the action of determining the milling slag falling speed feature according to the difference of the slag accumulation between multiple adjacent video image frames within the preset duration is performed. In this way, the accuracy of the milling slag falling speed feature can be improved.

[0082] In some possible implementation manners, the solution provided in this embodiment may further include step S210.

[0083] Step S210: Process the first video image through the milling position feature extraction module of the pre-trained milling strategy prediction model to obtain the operation position feature of the milling head of the milling equipment relative to the heading face.

[0084] In this embodiment, the milling position feature extraction module may identify the position of the milling head in the first video image to determine the operation position feature of the milling head of the milling equipment relative to the heading face.

[0085] For example, in the case where there is no bench milling in the tunnel, the operation position feature may represent the lifting height of the current milling head relative to the bottom of the tunnel or the relative height of the current milling head relative to the entire heading face. In the case where there is bench milling in the tunnel, the operation position feature may represent the lifting height of the milling head relative to the bench surface where the current milling equipment is located.

[0086] Specifically, during the milling operation, the lower the milling position, the more concentrated the position where the slag scatters, and the higher the milling position, the more dispersed the position where the slag scatters. When determining the milling slag falling speed feature according to the difference of the slag accumulation between multiple adjacent video image frames within the preset duration, it is recognized and processed based on 2D image frames, and the scattered distribution of the slag will have a certain influence on determining the difference of the slag accumulation between adjacent video image frames.

[0087] In addition, the milling operation is usually carried out from bottom to top on the heading face to avoid the influence of slag accumulation on the lower milling operation. In this case, for relatively loose rock formations or coatings, or rock formations with low strength, the lower the milling position, the more slag will fall due to the natural fall of the upper part of the rock formation in addition to the slag formed by the crushing of the milling head itself during the milling slag falling, which will also affect the judgment of the milling slag falling speed feature.

[0088] Therefore, in this embodiment, in step S123, the difference in slag accumulation between multiple adjacent video image frames and the operation position characteristics can be combined to determine the milling slag removal speed characteristics, thereby reducing the influence of the milling operation position on the recognition accuracy of the milling slag removal speed characteristics.

[0089] In some possible implementation manners, the solution provided in this embodiment may further include step S310.

[0090] Step S310: Obtain the working parameters of the milling head of the milling equipment, where the working parameters of the milling head include at least one of milling width, milling head diameter, number of cutter heads, current rotation speed, and torque.

[0091] In step S140, the pre-recorded rock layer exploration data, the working parameters of the milling head, and the milling slag removal speed characteristics can be fused to obtain fused characteristics.

[0092] Specifically, when the milling head performs milling operations with different working parameters, the milling slag removal speed may be different. For example, for the same rock layer, the greater the rotation speed of the milling head, the faster the milling slag removal speed. In this case, in order for the fused characteristics to better reflect the state of the current rock layer, the current working parameters of the milling head can be added to the fused characteristics. In this way, when predicting the milling step depth, the influence caused by different working parameters of the milling head can be balanced, so as to more accurately determine the milling step depth.

[0093] Furthermore, in some possible implementation manners, the method further includes the following steps.

[0094] Step S410: Input the fused characteristics into the milling parameter adjustment module of the milling strategy prediction model for processing to obtain the milling parameter adjustment result output by the milling parameter adjustment module.

[0095] Step S420: Apply the milling parameter adjustment result to the milling operation of the milling head.

[0096] In this embodiment, the fused characteristics can accurately reflect the state of the current rock layer. After obtaining the fused characteristics, the corresponding milling parameter adjustment result can be determined according to the fused characteristics, and thus applied to the milling operation of the milling head.

[0097] For example, when the rock layer strength reflected by the fused characteristics is relatively large, at the same power, the rotation speed of the milling head can be reduced and the torque can be increased to improve the milling force. When the rock layer strength reflected by the fused characteristics is relatively small, the torque of the milling head can be reduced and the rotation speed can be increased to improve the milling speed.

[0098] In some possible implementations, the method further includes the following steps.

[0099] Step S510: Input the fusion feature into the slag dropping speed warning module of the milling excavation strategy prediction model for processing to obtain a slag dropping speed prediction threshold.

[0100] Step S520: Detect whether the slag dropping speed corresponding to the milling excavation slag dropping speed feature reaches the slag dropping speed prediction threshold. If so, output a risk warning prompt.

[0101] Specifically, during milling excavation operations, if the slag dropping speed is too high, it is possible that there are certain structural problems in the rock formation itself, resulting in the disintegration of the rock formation itself, which may lead to construction accidents. It is necessary to issue a warning prompt when it is found that the milling excavation slag dropping speed is too fast. Different types and states of rock formations require different warning thresholds. Therefore, in this embodiment, after obtaining the fusion feature that can represent the current rock formation state, the corresponding slag dropping speed prediction threshold can be determined according to this fusion feature. And when it is detected that the slag dropping speed reaches the slag dropping speed prediction threshold, a risk warning prompt is output, thereby reducing the risk of construction accidents.

[0102] In some possible implementations, the method further includes the following steps.

[0103] Step S610: Determine the corresponding dust reduction operation control parameters according to the milling excavation slag dropping speed feature.

[0104] Step S620: Control the dust reduction equipment to perform dust reduction operations according to the dust reduction operation control parameters.

[0105] Specifically, during milling excavation operations, when the milling excavation equipment crushes the rock formation, a large amount of dust may be generated. The faster the milling excavation slag dropping speed, the greater the dust. Therefore, after determining the milling excavation slag dropping speed feature, the corresponding dust reduction operation control parameters can be determined according to the milling excavation slag dropping speed feature, and then the dust reduction equipment is controlled to perform dust reduction operations according to the dust reduction operation control parameters. In this way, the operation of the dust reduction equipment is linked with the milling excavation operation, improving the dust reduction efficiency and reducing the dust reduction energy consumption.

[0106] Please refer to Figure 4 , Figure 4 which Figure 1 is a block diagram of the data processing device 100 shown. The data processing device 100 includes a mechanical excavation device 110 for image recognition, a machine-readable storage medium 120, and a processor 130.

[0107] The components of the machine-readable storage medium 120 and the processor 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The image recognition mechanical excavation device 110 includes at least one software function module that can be stored in the machine-readable storage medium 120 in the form of software or firmware or solidified in the operating system (OS) of the data processing device 100. The processor 130 is used to execute the executable modules stored in the machine-readable storage medium 120, such as the software function modules and computer programs included in the image recognition mechanical excavation device 110.

[0108] The machine-readable storage medium 120 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The machine-readable storage medium 120 is used to store a program, and the processor 130 executes the program after receiving the execution instruction / executable the mechanical excavation method of image recognition provided in this embodiment.

[0109] The processor 130 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0110] Please refer to Figure 5, this embodiment also provides a mechanical excavation device 110 for image recognition. The mechanical excavation device 110 for image recognition includes at least one functional module that can be stored in a machine-readable storage medium 120 in software form. Functionally, the mechanical excavation device 110 for image recognition can include a video acquisition module 111, a feature extraction module 112, a feature fusion module 113, a step prediction module 114, and an operation instruction module 115.

[0111] The video acquisition module 111 is used to acquire the first video image of the heading face during the operation of the milling excavation equipment collected by the image acquisition device.

[0112] In this embodiment, the video acquisition module 111 can be used to execute Figure 2 the step S110 shown. For the specific description of the video acquisition module 111, reference can be made to the description of the step S110.

[0113] The feature extraction module 112 is used to process the first video image through the milling slag falling speed feature extraction module of the pre-trained milling excavation strategy prediction model to obtain the milling slag falling speed feature.

[0114] In this embodiment, the feature extraction module 112 can be used to execute Figure 2 the step S120 shown. For the specific description of the feature extraction module 112, reference can be made to the description of the step S120.

[0115] The feature fusion module 113 is used to fuse the pre-entered rock stratum exploration data with the milling slag falling speed feature to obtain a fusion feature.

[0116] In this embodiment, the feature fusion module 113 can be used to execute Figure 2 the step S130 shown. For the specific description of the feature fusion module 113, reference can be made to the description of the step S130.

[0117] The step prediction module 114 is used to input the fusion feature into the step depth prediction module of the milling excavation strategy prediction model to obtain the milling excavation step depth prediction value input by the step depth prediction module.

[0118] In this embodiment, the step prediction module 114 can be used to execute Figure 2 the step S140 shown. For the specific description of the step prediction module 114, reference can be made to the description of the step S140.

[0119] The operation instruction module 115 is used to send the milling excavation step depth prediction value to the milling excavation operator and / or the milling excavation equipment to instruct the milling excavation operator and / or the milling excavation equipment to perform the milling excavation operation according to the milling excavation step depth prediction value.

[0120] In this embodiment, the operation instruction module 115 can be used to execute Figure 2 the steps S150 shown. For the specific description of the operation instruction module 115, reference can be made to the description of the steps S150.

[0121] In summary, the present application provides a mechanical excavation method and device for image recognition. The first video image of the face during the operation of the milling excavation device is collected by an image acquisition device, and the milling excavation slag falling speed feature during the milling excavation process is determined through image recognition by a pre-established milling excavation strategy prediction model. Together with the pre-recorded rock stratum exploration data, the appropriate milling excavation step depth prediction value is determined. In this way, the milling excavation step depth can be adjusted in a timely manner when facing different rock stratum states during the milling excavation process, and the construction efficiency can be improved while ensuring construction safety.

[0122] In the embodiments provided by the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the drawings show the possible architectures, functions, and operations of the device, method, and computer program product according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0123] In addition, in each embodiment of the present application, the various functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0124] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0125] It should be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. Without further limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0126] As described above, these are only various implementation manners of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art in the technical field disclosed by this application can easily think of changes or substitutions within the technical scope disclosed by this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A mechanical excavation method using image recognition, characterized in that: The method comprises: Acquire a first video image of the tunnel face during the operation of the milling and excavation equipment by an image acquisition device; The first video image is processed by a milling and digging slag falling speed feature extraction module of a pre-trained milling and digging strategy prediction model to obtain a milling and digging slag falling speed feature; The pre-recorded rock formation exploration data is integrated with the milling and excavation debris velocity characteristics to obtain a fusion feature; Inputting the fusion feature into the step depth prediction module of the milling and digging strategy prediction model to obtain the milling and digging step depth prediction value input by the step depth prediction module; Sending the milling step depth prediction value to a milling operator and / or the milling equipment to instruct the milling operator and / or the milling equipment to perform the milling operation according to the milling step depth prediction value; The step of processing the first video image by the milling and digging slag velocity feature extraction module of the pre-trained milling and digging strategy prediction model to obtain the milling and digging slag velocity feature includes: Acquire a plurality of video image frames of the first video image; The milling and excavation slag speed feature extraction module performs content recognition on the plurality of video images respectively to determine the slag deposits in each of the video image frames; The milling and excavation slag velocity characteristics are determined based on the difference in the slag deposits between a plurality of adjacent video image frames within a preset time length.

2. The method according to claim 1, characterized in that The step of performing content recognition on the plurality of video images respectively by the milling and excavation slag falling speed feature extraction module to determine the slag falling deposits in each of the video image frames comprises: The content of the plurality of video images is respectively recognized by the milling and digging slag speed feature extraction module, and the slag accumulation in each of the video image frames is determined, and the working state of the milling and digging head of the milling and digging equipment is determined; the working state includes an operating state and a non-operating state; The step of determining the milling and excavation slag velocity characteristics according to the difference of the slag deposits between a plurality of adjacent video image frames within a preset time period comprises: When it is detected that the milling head is in the working state for a preset time, the milling and excavating slag speed characteristics are determined according to the difference of the slag deposits between a plurality of adjacent video image frames within the preset time.

3. The method according to claim 1, characterized in that The method further comprises: The first video image is processed by a milling position feature extraction module of a pre-trained milling strategy prediction model to obtain an operating position feature of a milling head of the milling equipment relative to the tunnel face; The step of determining the milling and excavation slag velocity characteristics according to the difference of the slag deposits between a plurality of adjacent video image frames within a preset time period comprises: The milling and excavation slag speed characteristics are determined based on the differences in the slag deposits between a plurality of adjacent video image frames within a preset time length and the operating position characteristics.

4. The method according to claim 1, characterized in that: The method further comprises: Acquire working parameters of the milling head of the milling equipment, wherein the working parameters of the milling head include at least one of a milling width, a milling head diameter, a number of cutter heads, a current rotation speed, and a torque; The step of fusing the pre-recorded rock formation exploration data with the milling and excavation slag velocity characteristics to obtain a fusion characteristic comprises: The pre-recorded rock formation exploration data, the working parameters of the milling head and the milling slag falling speed characteristics are fused to obtain a fusion feature.

5. The method according to claim 4, characterized in that The method further comprises: Inputting the fusion feature into the milling and digging parameter adjustment module of the milling and digging strategy prediction model for processing, and obtaining the milling and digging parameter adjustment result output by the milling and digging parameter adjustment module; The adjustment result of the milling parameters is applied to the milling operation of the milling head.

6. The method according to claim 4, characterized in that The method further comprises: The fusion feature is input into the slag falling speed warning module of the milling and digging strategy prediction model for processing to obtain a slag falling speed prediction threshold; Detecting whether the slag falling speed corresponding to the milling and excavation slag falling speed characteristic reaches the slag falling speed prediction threshold; If so, a risk warning prompt is output.

7. The method according to claim 1, characterized in that The method further comprises: Determine corresponding dust reduction operation control parameters according to the milling and excavation slag falling speed characteristics; The dust reduction equipment is controlled to perform the dust reduction operation according to the dust reduction operation control parameters.

8. A mechanical excavation device for image recognition, characterized in that: The image recognition mechanical excavation device comprises: A video acquisition module, used to acquire a first video image of the tunnel face during the operation of the milling and excavation equipment acquired by an image acquisition device; A feature extraction module, used to process the first video image through a milling and digging slag falling speed feature extraction module of a pre-trained milling and digging strategy prediction model to obtain a milling and digging slag falling speed feature; A feature fusion module is used to fuse the pre-recorded rock formation exploration data with the milling and excavation slag velocity feature to obtain a fusion feature; A step prediction module, used for inputting the fusion feature into the step depth prediction module of the milling and digging strategy prediction model to obtain the milling and digging step depth prediction value input by the step depth prediction module; An operation instruction module, used for sending the predicted value of the milling step depth to the milling operator and / or the milling equipment, so as to instruct the milling operator and / or the milling equipment to perform the milling operation according to the predicted value of the milling step depth; Wherein, the feature extraction module is specifically used for: Acquire a plurality of video image frames of the first video image; The milling and excavation slag speed feature extraction module performs content recognition on the plurality of video images respectively to determine the slag deposits in each of the video image frames; The milling and excavation slag velocity characteristics are determined based on the difference in the slag deposits between a plurality of adjacent video image frames within a preset time length.

Images