Mechanical excavation method and device based on image recognition

Abstract

The invention provides a mechanical excavation method and device based on image recognition, and the method comprises the steps: collecting a first video image of a tunnel face in the operation process of milling and excavation equipment through image collection equipment, carrying out the image recognition through a pre-milling and excavation strategy prediction model, and determining the milling and excavation slag falling speed characteristics in the milling and excavation process; according to the method, the proper milling and digging stepping depth predicted value is determined by combining the pre-recorded rock stratum exploration data, so that the milling and digging stepping depth can be adjusted in time when different rock stratum states are faced in the milling and digging process, and the construction efficiency is improved under the condition that the construction safety is ensured.

Description

Technical Field

[0001] The present application relates to the field of AI-based image recognition processing, and more specifically, to a mechanical excavation method and device for image recognition. Background Art

[0002] In some tunnel construction operations, milling equipment is used for tunnel excavation to reduce the impact of excavation on surrounding rock formations or ground facilities. During the milling operation, the milling equipment performs milling operations on the face through the milling head. After milling to a certain depth, support is carried out, and then further milling is carried out. Support and milling operations are carried out alternately. However, the state of the rock formations at different locations may be different, and the state of the rock formations directly affects the milling step depth. If the milling step depth is too small, the support and milling operations are alternated too frequently, affecting the construction speed; if the milling depth is too large, there may be construction safety hazards if support is not timely. How to timely and accurately adjust the milling step depth adaptively has become an urgent problem to be solved. Summary of the invention

[0003] In order to overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide a mechanical excavation method for image recognition, the method comprising: Acquire a first video image of the tunnel face during the operation of the milling and excavation equipment by an image acquisition device; The first video image is processed by a milling and digging slag falling speed feature extraction module of a pre-trained milling and digging strategy prediction model to obtain a milling and digging slag falling speed feature; The pre-recorded rock formation exploration data is integrated with the milling and excavation debris velocity characteristics to obtain a fusion feature; Inputting the fusion feature into the step depth prediction module of the milling and digging strategy prediction model to obtain the milling and digging step depth prediction value input by the step depth prediction module; The predicted value of the milling step depth is sent to the milling operator and / or the milling equipment to instruct the milling operator and / or the milling equipment to perform the milling operation according to the predicted value of the milling step depth.

[0004] In some possible implementations, the step of processing the first video image by a milling and digging slag velocity feature extraction module of a pre-trained milling and digging strategy prediction model to obtain the milling and digging slag velocity feature includes: Acquire a plurality of video image frames of the first video image; The milling and excavation slag speed feature extraction module performs content recognition on the plurality of video images respectively to determine the slag deposits in each of the video image frames; The milling and excavation slag velocity characteristics are determined based on the difference in the slag deposits between a plurality of adjacent video image frames within a preset time length.

[0005] In some possible implementations, the step of performing content recognition on the plurality of video images respectively by the milling and excavation slag velocity feature extraction module to determine the slag deposits in each of the video image frames includes: The content of the plurality of video images is respectively recognized by the milling and digging slag speed feature extraction module, and the slag accumulation in each of the video image frames is determined, and the working state of the milling and digging head of the milling and digging equipment is determined; the working state includes an operating state and a non-operating state; The step of determining the milling and excavation slag velocity characteristics according to the difference of the slag deposits between a plurality of adjacent video image frames within a preset time period comprises: When it is detected that the milling head is in the working state for a preset time, the milling and excavating slag speed characteristics are determined according to the difference of the slag deposits between a plurality of adjacent video image frames within the preset time.

[0006] In some possible implementations, the method further includes: The first video image is processed by a milling position feature extraction module of a pre-trained milling strategy prediction model to obtain an operating position feature of a milling head of the milling equipment relative to the tunnel face; The step of determining the milling and excavation slag velocity characteristics according to the difference of the slag deposits between a plurality of adjacent video image frames within a preset time period comprises: The milling and excavation slag speed characteristics are determined based on the differences in the slag deposits between a plurality of adjacent video image frames within a preset time length and the operating position characteristics.

[0007] In some possible implementations, the method further includes: Acquire working parameters of the milling head of the milling equipment, wherein the working parameters of the milling head include at least one of a milling width, a milling head diameter, a number of cutter heads, a current rotation speed, and a torque; The step of fusing the pre-recorded rock formation exploration data with the milling and excavation slag velocity characteristics to obtain a fusion characteristic comprises: The pre-recorded rock formation exploration data, the working parameters of the milling head and the milling slag falling speed characteristics are fused to obtain a fusion feature.

[0008] In some possible implementations, the method further includes: Inputting the fusion feature into the milling and digging parameter adjustment module of the milling and digging strategy prediction model for processing, and obtaining the milling and digging parameter adjustment result output by the milling and digging parameter adjustment module; The adjustment result of the milling parameters is applied to the milling operation of the milling head.

[0009] In some possible implementations, the method further includes: Determine corresponding dust reduction operation control parameters according to the milling and excavation slag falling speed characteristics; The dust reduction equipment is controlled to perform the dust reduction operation according to the dust reduction operation control parameters.

[0010] In some possible implementations, the method further includes: The fusion feature is input into the slag falling speed warning module of the milling and digging strategy prediction model for processing to obtain a slag falling speed prediction threshold; Detecting whether the slag falling speed corresponding to the milling and excavation slag falling speed characteristic reaches the slag falling speed prediction threshold; If so, a risk warning prompt is output.

[0011] Another object of the present application is to provide an image recognition mechanical excavation device, the image recognition mechanical excavation device comprising: A video acquisition module, used to acquire a first video image of the tunnel face during the operation of the milling and excavation equipment acquired by an image acquisition device; A feature extraction module, used to process the first video image through a milling and digging slag falling speed feature extraction module of a pre-trained milling and digging strategy prediction model to obtain a milling and digging slag falling speed feature; A feature fusion module is used to fuse the pre-recorded rock formation exploration data with the milling and excavation slag velocity feature to obtain a fusion feature; A step prediction module, used for inputting the fusion feature into the step depth prediction module of the milling and digging strategy prediction model to obtain the milling and digging step depth prediction value input by the step depth prediction module; The operation instruction module is used to send the milling step depth prediction value to the milling operator and / or the milling equipment to instruct the milling operator and / or the milling equipment to perform the milling operation according to the milling step depth prediction value.

[0012] Compared with the prior art, this application has the following beneficial effects: The present application provides a mechanical excavation method and device for image recognition, which uses an image acquisition device to collect the first video image of the face during the operation of the milling and excavation equipment, and uses a pre-prepared milling and excavation strategy prediction model to perform image recognition to determine the milling and excavation slag velocity characteristics during the milling and excavation process, and combines the pre-entered rock formation exploration data to determine a suitable milling and excavation step depth prediction value. In this way, the milling and excavation step depth can be adjusted in time when facing different rock formation conditions during the milling and excavation process, thereby improving construction efficiency while ensuring construction safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 A schematic diagram of a mechanical excavation system for image recognition provided in an embodiment of the present application; Figure 2 A schematic diagram of the steps of the mechanical excavation system method for image recognition provided in an embodiment of the present application; Figure 3 A schematic diagram of a milling and digging scenario provided in an embodiment of the present application; Figure 4 A schematic diagram of a data processing device provided in an embodiment of the present application; Figure 5 A schematic diagram of the functional modules of a mechanical excavation device with image recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0016] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0017] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0018] In the description of the present application, it should be noted that the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0019] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0020] See also Figure 1 , Figure 1 A schematic diagram of an image recognition mechanical excavation system provided in this embodiment, the system may include a data processing device 100 and at least one image acquisition device 200, which acquires a first video image of the face during the operation of the milling and excavation equipment through the image acquisition device, and determines the milling and excavation slag velocity characteristics during the milling and excavation process through image recognition through a pre-prepared milling and excavation strategy prediction model, and determines a suitable milling and excavation step depth prediction value in combination with pre-recorded rock formation exploration data.

[0021] Specifically, the data processing device 100 can be used to execute a mechanical excavation method for image recognition provided in this embodiment, see Figure 2 , the method may include the following steps.

[0022] Step S110, obtaining a first video image of the tunnel face during the operation of the milling and excavation equipment through an image acquisition device.

[0023] In this embodiment, the image acquisition device can be arranged on the milling device, or can also be arranged on the top of the tunnel. The image acquisition device can acquire the first video image of the tunnel face during the operation of the milling device, that is, acquire the image of the rock face being milled.

[0024] Step S120, processing the first video image by a milling and excavation slag falling speed feature extraction module of a pre-trained milling and excavation strategy prediction model to obtain a milling and excavation slag falling speed feature.

[0025] See also Figure 3 , Figure 3 Schematic diagram of the milling process of the milling device 600, wherein the milling device 600 may be a cantilever excavation device provided with a milling head, and the milling device may use the milling head to mill the rock formation of the tunnel face 700. During the milling process, the milling head rotates to break the rock formation, and the broken rock formation debris falls downward and accumulates at the bottom of the tunnel face to form a debris accumulation 800.

[0026] In this embodiment, the data processing device 100 can receive the first video image captured by the image acquisition device 200, and input the first video image into the pre-trained milling and excavation strategy prediction model, and process the first video image through the milling and excavation slag speed feature extraction module of the milling and excavation strategy prediction model to obtain the milling and excavation slag speed feature that characterizes the slag speed during the milling and excavation process.

[0027] Among them, the milling slag velocity characteristics can characterize the rock state of the current excavation location to a certain extent. For example, when the milling head has the same rotation speed, the slag velocity generated by the location with greater rock strength is relatively small, while the slag velocity generated by the location with less rock strength is relatively large.

[0028] Step S130, merging the pre-recorded rock formation exploration data with the milling and excavation debris velocity characteristics to obtain a fusion feature.

[0029] In this embodiment, during the preliminary exploration or construction of the tunnel, drilling and sampling of the rock formation are performed to obtain rock formation exploration data, which may include data such as the type, elastic modulus, Poisson's ratio, and rock formation strength of rock formations at different locations.

[0030] In this embodiment, the rock formation exploration data and the milling and excavation debris velocity characteristics may be fused, for example, the milling and excavation debris velocity characteristics at the same position are spliced ​​with the rock formation exploration data to form the fused characteristics.

[0031] Step S140: input the fusion feature into the step depth prediction module of the milling and digging strategy prediction model to obtain the milling and digging step depth prediction value input by the step depth prediction module.

[0032] In this embodiment, considering that the rock formation exploration data can only characterize the general characteristics of the rock formation, and the rock formation conditions at different locations may be different, therefore, based on the rock formation exploration data, combined with the currently extracted milling and excavation debris velocity characteristics, a suitable milling and excavation step depth prediction value is predicted.

[0033] For example, in one example, a basic step depth can be determined based on the characteristics of the rock formation type in the fusion feature, and then an adjusted step depth can be determined based on the characteristics of the milling and excavation debris falling speed in the fusion feature, and the basic step depth and the adjusted step depth are weighted and summed to obtain a predicted value of the milling and excavation step depth.

[0034] For another example, the fused features may be input into the depth prediction module for feature fitting and regression prediction to obtain an output milling step depth prediction value.

[0035] Step S150: sending the predicted value of the milling step depth to the milling operator and / or the milling equipment to instruct the milling operator and / or the milling equipment to perform the milling operation according to the predicted value of the milling step depth.

[0036] In this embodiment, after obtaining the milling step depth prediction value, it can be sent to the milling operator so that he can control the milling equipment to perform the milling operation according to the milling step depth prediction value, or the milling step depth prediction value can be sent to the milling equipment so that it can automatically perform the milling operation according to the milling step depth prediction value.

[0037] Based on the above design, in the method provided in the present application, the first video image of the face during the operation of the milling and excavation equipment is collected by an image acquisition device, and the milling and excavation slag velocity characteristics during the milling and excavation process are determined by image recognition through a pre-prepared milling and excavation strategy prediction model. The suitable milling and excavation step depth prediction value is determined together with the pre-entered rock formation exploration data. In this way, the milling and excavation step depth can be adjusted in time when facing different rock formation conditions during the milling and excavation process, thereby improving construction efficiency while ensuring construction safety.

[0038] In some possible implementations, step S120 may include the following sub-steps.

[0039] Step S121: Acquire multiple video image frames of the first video image.

[0040] Step S122, performing content recognition on the plurality of video images respectively through the milling and excavation slag velocity feature extraction module to determine the slag deposits in each of the video image frames.

[0041] Step S123, determining the milling and excavation slag velocity characteristics according to the difference of the slag deposits between a plurality of adjacent video image frames within a preset time length.

[0042] Specifically, in this embodiment, the milling and excavation slag speed feature extraction module can be configured to identify slag deposits in the first video image, and determine the milling and excavation slag speed feature according to changes in the slag deposits in multiple video image frames.

[0043] Optionally, in this embodiment, a plurality of video image frames may be acquired at the same sampling interval, and then a difference value of the slag accumulation between every two adjacent video image frames may be acquired, and the difference value may be a coverage range of the identified slag accumulation or a slag volume determined according to the coverage range of the slag accumulation. Then, the milling and excavation slag velocity feature is determined according to an average value of the difference values ​​of the slag accumulation between the plurality of video image frames.

[0044] Furthermore, in step S122, the milling and excavation slag speed feature extraction module can be used to perform content recognition on the plurality of video images respectively, to determine the slag accumulation in each of the video image frames, and also to determine the working state of the milling and excavation head of the milling and excavation equipment. The working state includes an operating state and a non-operating state.

[0045] For example, when it is identified that the milling head is in a rotating state and the slag accumulation increases, the working state is determined to be an operating state. When it is identified that the milling head is not rotating, and / or the milling head is rotating but the slag accumulation does not increase, the working state is determined to be a non-working state.

[0046] In step S123, when it is detected that the milling head is in working state for a preset time, the milling and excavating slag speed characteristic can be determined according to the difference of the slag deposits between a plurality of adjacent video image frames within the preset time.

[0047] Specifically, in the short time when the milling head just contacts the rock formation of the face, it is not yet in a stable milling state (i.e., in a startup state), and the milling slag speed at this time will be significantly lower than the milling slag speed during stable milling. In order to reduce the influence of the startup state on the identification of the milling slag speed, and avoid the influence of the non-working state on the identification of the milling slag speed, in this embodiment, when it is detected that the milling head is in a working state for a preset time, it can be considered that the milling action is in a stable milling state, and then the action of determining the milling slag speed characteristics based on the difference in the slag deposits between multiple adjacent video image frames within the preset time is executed. In this way, the accuracy of the milling slag speed characteristics can be improved.

[0048] In some possible implementations, the solution provided in this embodiment may further include step S210.

[0049] Step S210: Processing the first video image by a milling position feature extraction module of a pre-trained milling strategy prediction model to obtain operating position features of the milling head of the milling equipment relative to the tunnel face.

[0050] In this embodiment, the milling position feature extraction module can identify the position of the milling head in the first video image, and determine the operating position features of the milling head of the milling equipment relative to the tunnel face.

[0051] For example, in the case where there is no step milling in the tunnel, the operating position feature can represent the lifting height of the current milling head relative to the bottom of the tunnel, or the relative height of the current milling head relative to the entire tunnel face. In the case where there is step milling in the tunnel, the operating position feature can represent the lifting height of the milling head relative to the step surface where the current milling equipment is located.

[0052] Specifically, when performing milling and digging operations, the lower the milling and digging position is, the more concentrated the positions of the slag scattering are, and the higher the milling and digging position is, the more dispersed the positions of the slag scattering are. When determining the milling and digging slag velocity characteristics based on the difference of the slag accumulation between multiple adjacent video image frames within a preset time, recognition processing is performed based on 2D image frames, and the slag scattering distribution will have a certain influence on determining the difference of the slag accumulation between adjacent video image frames.

[0053] In addition, milling operations usually mill the tunnel face from bottom to top to avoid the accumulation of debris affecting the lower milling operations. In this case, for looser rock layers or coatings, or rock layers with lower strength, the lower the milling position, the more debris will fall from the milling head itself during milling. In addition, there will also be debris that falls naturally from the upper part of the rock layer due to gravity, which will also affect the judgment of the characteristics of the milling debris speed.

[0054] Therefore, in this embodiment, in step S123, the milling and excavation slag speed characteristics can be determined in combination with the differences in the slag deposits between multiple adjacent video image frames and the operating position characteristics, thereby reducing the influence of the milling and excavation operating position on the accuracy of identifying the milling and excavation slag speed characteristics.

[0055] In some possible implementations, the solution provided in this embodiment may further include steps S310, Step S310, obtaining the working parameters of the milling head of the milling equipment, wherein the working parameters of the milling head include at least one of the milling width, the diameter of the milling head, the number of cutter bits, the current rotation speed, and the torque.

[0056] In step S140, the pre-recorded rock formation exploration data, the working parameters of the milling head and the milling and slag falling speed characteristics may be fused to obtain a fusion feature.

[0057] Specifically, when the milling head uses different working parameters to perform milling operations, the milling and slag dropping speed may be different. For example, for the same rock formation, the greater the rotation speed of the milling head, the faster the milling and slag dropping speed. In this case, in order to make the fusion feature better reflect the state of the current rock formation, the current milling head working parameters can be added to the fusion feature. In this way, when predicting the milling step depth, the influence of different milling head working parameters is balanced, so as to more accurately determine the milling step depth.

[0058] Furthermore, in some possible implementations, the method further includes the following steps.

[0059] Step S410: input the fusion feature into the milling and digging parameter adjustment module of the milling and digging strategy prediction model for processing, and obtain the milling and digging parameter adjustment result output by the milling and digging parameter adjustment module.

[0060] Step S420: applying the milling and digging parameter adjustment result to the milling and digging operation of the milling and digging head.

[0061] In this embodiment, the fusion feature can accurately reflect the current state of the rock formation. After the fusion feature is obtained, the corresponding milling parameter adjustment result can be determined according to the fusion feature, so as to be applied to the milling operation of the milling head.

[0062] For example, when the rock formation strength reflected by the fusion feature is relatively large, at the same power, the rotation speed of the milling head can be reduced, the torque can be increased, and the milling force can be increased. When the rock formation strength reflected by the fusion feature is relatively small, the torque of the milling head can be reduced, the rotation speed can be increased, and the milling speed can be increased.

[0063] In some possible implementations, the method further includes the following steps.

[0064] Step S510: input the fusion feature into the slag falling speed warning module of the milling and excavation strategy prediction model for processing to obtain a slag falling speed prediction threshold.

[0065] Step S520, detecting whether the slag falling speed corresponding to the milling slag falling speed characteristic reaches the slag falling speed prediction threshold. If so, outputting a risk warning prompt.

[0066] Specifically, if the slag falling speed is too high during the milling and excavation operation, it is possible that there are certain structural problems in the rock formation itself, which may cause the rock formation itself to collapse, which may lead to construction accidents. It is necessary to issue an early warning prompt when it is found that the milling and excavation slag falling speed is too fast. Different types and states of rock formations require different early warning thresholds. Therefore, in this embodiment, after obtaining the fusion feature that can characterize the current state of the rock formation, the corresponding slag falling speed prediction threshold can be determined based on the fusion feature. And when it is detected that the slag falling speed reaches the slag falling speed prediction threshold, a risk warning prompt is output to reduce the risk of construction accidents.

[0067] In some possible implementations, the method further includes the following steps.

[0068] Step S610, determining corresponding dust reduction operation control parameters according to the milling and excavation debris falling speed characteristics.

[0069] Step S620: Control the dust reduction equipment to perform the dust reduction operation according to the dust reduction operation control parameters.

[0070] Specifically, during the milling and excavation operation, the milling and excavation equipment may generate a large amount of dust when milling and excavating the rock layer. The faster the milling and excavation slag falling speed, the greater the dust. Therefore, after determining the milling and excavation slag falling speed characteristics, the corresponding dust reduction operation control parameters can be determined according to the milling and excavation slag falling speed characteristics, and then the dust reduction equipment is controlled to perform the dust reduction operation according to the dust reduction operation control parameters. In this way, the operation of the dust reduction equipment is linked with the milling and excavation operation, the dust reduction efficiency is improved, and the dust reduction energy consumption is reduced.

[0071] Please refer to Figure 4 , Figure 4 yes Figure 1 The block diagram of the data processing device 100 is shown. The data processing device 100 comprises a mechanical excavation device 110 for image recognition, a machine-readable storage medium 120, and a processor 130.

[0072] The components of the machine-readable storage medium 120 and the processor 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The image recognition mechanical excavation device 110 includes at least one software function module that can be stored in the machine-readable storage medium 120 in the form of software or firmware or solidified in the operating system (OS) of the data processing device 100. The processor 130 is used to execute the executable modules stored in the machine-readable storage medium 120, such as the software function modules and computer programs included in the image recognition mechanical excavation device 110.

[0073] The machine-readable storage medium 120 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The machine-readable storage medium 120 is used to store a program, and the processor 130 executes the program after receiving the execution instruction / executable the mechanical excavation method of image recognition provided in this embodiment.

[0074] The processor 130 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0075] Please refer to Figure 5 This embodiment further provides an image recognition mechanical excavation device 110, which includes at least one functional module that can be stored in a machine-readable storage medium 120 in the form of software. From a functional perspective, the image recognition mechanical excavation device 110 may include a video acquisition module 111, a feature extraction module 112, a feature fusion module 113, a step prediction module 114, and an operation instruction module 115.

[0076] The video acquisition module 111 is used to acquire a first video image of the tunnel face captured by an image acquisition device during the operation of the milling and excavation equipment.

[0077] In this embodiment, the video acquisition module 111 can be used to perform Figure 2 As shown in step S110, for a detailed description of the video acquisition module 111, please refer to the description of step S110.

[0078] The feature extraction module 112 is used to process the first video image through the milling and excavation slag falling speed feature extraction module 112 of the pre-trained milling and excavation strategy prediction model to obtain the milling and excavation slag falling speed feature.

[0079] In this embodiment, the feature extraction module 112 can be used to perform Figure 2 As shown in step S120, for a detailed description of the feature extraction module 112, please refer to the description of step S120.

[0080] The feature fusion module 113 is used to fuse the pre-recorded rock formation exploration data with the milling and excavation debris velocity feature to obtain a fusion feature.

[0081] In this embodiment, the feature fusion module 113 can be used to perform Figure 2 As shown in step S130, for a detailed description of the feature fusion module 113, please refer to the description of step S130.

[0082] The step prediction module 114 is used to input the fusion feature into the step depth prediction module of the milling and digging strategy prediction model to obtain the milling and digging step depth prediction value input by the step depth prediction module.

[0083] In this embodiment, the step prediction module 114 can be used to perform Figure 2 As shown in step S140 , for a detailed description of the step prediction module 114 , please refer to the description of step S140 .

[0084] The operation instruction module 115 is used to send the milling step depth prediction value to the milling operator and / or the milling equipment to instruct the milling operator and / or the milling equipment to perform the milling operation according to the milling step depth prediction value.

[0085] In this embodiment, the operation instruction module 115 can be used to execute Figure 2 As shown in step S150, for a detailed description of the operation instruction module 115, please refer to the description of step S150.

[0086] To summarize, the present application provides a mechanical excavation method and device based on image recognition, which uses an image acquisition device to collect the first video image of the face during the operation of the milling and excavation equipment, and uses a pre-prepared milling and excavation strategy prediction model to perform image recognition to determine the milling and excavation slag velocity characteristics during the milling and excavation process, and combines the pre-entered rock formation exploration data to determine a suitable milling and excavation step depth prediction value. In this way, the milling and excavation step depth can be adjusted in time when facing different rock formation conditions during the milling and excavation process, thereby improving construction efficiency while ensuring construction safety.

[0087] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0088] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0089] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0090] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0091] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A mechanical excavation method using image recognition, characterized in that: The method comprises: Acquire a first video image of the tunnel face during the operation of the milling and excavation equipment by an image acquisition device; The first video image is processed by a milling and digging slag falling speed feature extraction module of a pre-trained milling and digging strategy prediction model to obtain a milling and digging slag falling speed feature; The pre-recorded rock formation exploration data is integrated with the milling and excavation debris velocity characteristics to obtain a fusion feature; Inputting the fusion feature into the step depth prediction module of the milling and digging strategy prediction model to obtain the milling and digging step depth prediction value input by the step depth prediction module; The predicted value of the milling step depth is sent to the milling operator and / or the milling equipment to instruct the milling operator and / or the milling equipment to perform the milling operation according to the predicted value of the milling step depth.

2. The method according to claim 1, characterized in that The step of processing the first video image by the milling and digging slag velocity feature extraction module of the pre-trained milling and digging strategy prediction model to obtain the milling and digging slag velocity feature includes: Acquire a plurality of video image frames of the first video image; The milling and excavation slag speed feature extraction module performs content recognition on the plurality of video images respectively to determine the slag deposits in each of the video image frames; The milling and excavation slag velocity characteristics are determined based on the difference in the slag deposits between a plurality of adjacent video image frames within a preset time length.

3. The method according to claim 2, characterized in that The step of performing content recognition on the plurality of video images respectively by the milling and excavation slag falling speed feature extraction module to determine the slag falling deposits in each of the video image frames comprises: The content of the plurality of video images is respectively recognized by the milling and digging slag speed feature extraction module, and the slag accumulation in each of the video image frames is determined, and the working state of the milling and digging head of the milling and digging equipment is determined; the working state includes an operating state and a non-operating state; The step of determining the milling and excavation slag velocity characteristics according to the difference of the slag deposits between a plurality of adjacent video image frames within a preset time period comprises: When it is detected that the milling head is in the working state for a preset time, the milling and excavating slag speed characteristics are determined according to the difference of the slag deposits between a plurality of adjacent video image frames within the preset time.

4. The method according to claim 2, characterized in that: The method further comprises: The first video image is processed by a milling position feature extraction module of a pre-trained milling strategy prediction model to obtain an operating position feature of a milling head of the milling equipment relative to the tunnel face; The step of determining the milling and excavation slag velocity characteristics according to the difference of the slag deposits between a plurality of adjacent video image frames within a preset time period comprises: The milling and excavation slag speed characteristics are determined based on the differences in the slag deposits between a plurality of adjacent video image frames within a preset time length and the operating position characteristics.

5. The method according to claim 1, characterized in that The method further comprises: Acquire working parameters of the milling head of the milling equipment, wherein the working parameters of the milling head include at least one of a milling width, a milling head diameter, a number of cutter heads, a current rotation speed, and a torque; The step of fusing the pre-recorded rock formation exploration data with the milling and excavation slag velocity characteristics to obtain a fusion characteristic comprises: The pre-recorded rock formation exploration data, the working parameters of the milling head and the milling slag falling speed characteristics are fused to obtain a fusion feature.

6. The method according to claim 5, characterized in that The method further comprises: Inputting the fusion feature into the milling and digging parameter adjustment module of the milling and digging strategy prediction model for processing, and obtaining the milling and digging parameter adjustment result output by the milling and digging parameter adjustment module; The adjustment result of the milling parameters is applied to the milling operation of the milling head.

7. The method according to claim 5, characterized in that The method further comprises: The fusion feature is input into the slag falling speed warning module of the milling and digging strategy prediction model for processing to obtain a slag falling speed prediction threshold; Detecting whether the slag falling speed corresponding to the milling and excavation slag falling speed characteristic reaches the slag falling speed prediction threshold; If so, a risk warning prompt is output.

8. The method according to claim 1, characterized in that: The method further comprises: Determine corresponding dust reduction operation control parameters according to the milling and excavation slag falling speed characteristics; The dust reduction equipment is controlled to perform the dust reduction operation according to the dust reduction operation control parameters.

9. A mechanical excavation device with image recognition, characterized in that: The image recognition mechanical excavation device comprises: A video acquisition module, used to acquire a first video image of the tunnel face during the operation of the milling and excavation equipment acquired by an image acquisition device; A feature extraction module, used to process the first video image through a milling and digging slag falling speed feature extraction module of a pre-trained milling and digging strategy prediction model to obtain a milling and digging slag falling speed feature; A feature fusion module is used to fuse the pre-recorded rock formation exploration data with the milling and excavation slag velocity feature to obtain a fusion feature; A step prediction module, used for inputting the fusion feature into the step depth prediction module of the milling and digging strategy prediction model to obtain the milling and digging step depth prediction value input by the step depth prediction module; The operation instruction module is used to send the milling step depth prediction value to the milling operator and / or the milling equipment to instruct the milling operator and / or the milling equipment to perform the milling operation according to the milling step depth prediction value.

Images